Data Analysis 101: How to Avoid Misinterpreting Stats

Deciphering and dissecting information holds supreme importance for individuals across the board, transcending the realm of scientific inquiry. For it is the mishandling of data interpretation that can yield fallacious deductions and misguided decision-making.

In this blog, my aim is to embolden those outside the scientific sphere with the understanding necessary to sidestep the most common pitfalls of data misinterpretation. We shall delve into the rudiments of data, foster a critical mindset, explore pragmatic approaches, and furnish invaluable insights on analyzing and effectively conveying data. With these tools in your possession, you will traverse the intricate landscape of data with confidence and make valuable contributions to a society steeped in data literacy. Together, let us unlock the potential that lies within the realm of data interpretation!

Understand the Context

The first step you must take when interpreting a graph or chart is to understand the context - no graph can be fully understood without it! Oftentimes, graphs are accompanied by text that provides a mathematical explanation of the trends that are being displayed. They put these here for a reason, to avoid the chance that a reader may be misled by any math they may have performed on the data involved.

In addition to the descriptors provided with graphs, it’s imperative that you can also define the problem or question the data aims to address. Familiarize yourself with the hypothesis being tested, the data source, and also the collection methods. One of the most lovely aspects of scientific papers is that this section is kept separate and labeled for this specific purpose. Though this section may not be as exciting or flashy as the efficient abstract, or the speculative conclusion, it’s one of the most important sections.

Develop a Critical Mindset

Even reputable journals, trusted friends, or large organizations, continually practice skepticism and question even the most professional-looking graphs and charts. Curiosity is a valuable resource. Seek additional information and emphasize learning as you go about your viewing. By adopting a critical mindset, you can avoid falling prey to misinformation or manipulation, even from seemingly reliable sources.

Note that anyone - including scientific institutions and well-known thought leaders - can accidentally have a bias or make mistakes. Scientists are still human and, even with the best intentions can make mistakes or pareidolia themselves into seeing patterns when they aren’t there. Recognize that even experts and scientific institutions can inadvertently introduce bias into their work, leading to skewed interpretations or conclusions. Understanding this human fallibility encourages humility in our own assessments and keeps us open to questioning assumptions and perspectives.

The large collection of word-heavy text that often accompanies graphs is placed there so that those with critical mindsets and a wealth of statistical reasoning can critically assess the data. Detailed explanations, methodological descriptions, and statistical reasoning provide transparency and allow readers to critically assess the data and conclusions. The absence of such information should be a red flag, signaling the need for further investigation before accepting the presented information.

Familiarize Yourself with Common Graphs and Charts

One of the best ways to avoid misinterpreting statistics and study findings is to take the time to learn the various sorts of graphs and charts. Daunting, I know, but once you have a basic understanding of why certain graphs are chosen over others, it will make it remarkably easier to interpret what’s going on. (I digress here, but the skill is also remarkably useful for mathematical reasoning. To be familiar enough with data to visualize it properly in one’s head is to have a better sense of numbers generally, which can aid with learning!)

To begin, check out this quick article by Piktochart.com that covers the 10 most common types of charts that are used. This guide does a wonderful job of explaining how they can be used to effectively communicate data. Once you have a decent grasp on the information there, solidify that information by navigating to r/DataIsBeautiful, a subreddit dedicated to making lovely graphs and charts. Bonus points if you take time to read comments on the images - there’s often some great conversation between stats folks explaining their reasoning or the data.

Pay Attention to Axes, Labels, and Units

It’s phenomenally easy to make a graph look way more exciting than the data it shows. Modifying the units to smaller measurements, taking the log of the output variables, or stretching an axis out, there are a wide variety of methods that scientists employ to make significant trends look more obvious. Though not intentionally misleading, the non-discerning reader could potentially see a trend as more pronounced than it truly is.

When viewing graphs, take time to understand the meaning of each axis and its units of measurement. Read and interpret those labels and their accompanying captions carefully. In addition, consider the scales and intervals used on the axes. If these labels aren’t present, or this is no explanation attached to the graph, tread carefully.

Identify Key Trends and Patterns

When viewing a graph, especially line graphs, bar graphs, or scatter plots, look for overall trends in the data (eg. increasing, decreasing, or fluctuating). The value of displaying data on charts is the opportunity it provides to compare them quickly and visually. When viewing a graph, ask yourself: from left to right, are the values increasing, decreasing, or staying the same, or is there seemingly no visual pattern?

Also, consider the implications and possible explanations for observed patterns - it’s not always as cut and dry as a graph may appear. Just because two variables correlate, does not mean that they necessarily exclusively affect each other. Look for the terms insignificant or significant – these are often included in the text to indicate whether a scientist has calculated that the variables being compared actually have a notable effect on each other. Also, look for a reference to a p-value. A p-value of less than 0.05, calculated using a bunch of math unnecessary for the scope of this article, is accepted as significant. In other words, this number means that there is only a 5% chance that this data happened randomly.

A note: An outlier for data purposes is a number that differs significantly from all other numbers of a particular group or set in experimentation. You may come across this as you’re reading through scientific papers. If a pattern is strong enough, and an unpredicted number could be explained by another factor, scientists might omit a result for simplicity’s sake. This isn’t necessarily something to be concerned about. If the writer provides a valid reason for the omission, proceed without worry.

Seek Additional Context and Verification

A graph or chart is only as good as the study it came from. Before accepting the trends in any given graphic, ask yourself where they got the information from. If a graph doesn’t have any sort of link, source, or, at the very least, a good explanation in the caption, this is a massive red flag.

Cross-reference the data with other reliable sources. If this graph is in contrast with every other piece of data on the subject, that’s another red flag. If you’re unsure about this, it can be incredibly helpful to read the comments associated - are people pointing out inconsistencies? Or are people arguing with each other about minutia in the data? More causes for concern. If, on the other hand, comments trend towards both positive and academic, there may be some merit to the content.

Note here that general comments can lead you in the right direction, but look for explanations or interpretations from experts in the field and prioritize reading these. Also, consider the timeframe and potential changes that might have affected the data. Generally speaking, more recent iterations of an experiment and more recently administered studies are more trustworthy.

Avoid Jumping to Conclusions

In any field of study or decision-making process, making assumptions without adequate evidence can lead to flawed conclusions and misguided actions. You must base your conclusions about an article on the soundest reasoning possible, and take time to consider whether you’re basing your opinion on preconceived notions or personal biases.

Recognize the limitations and uncertainties in the data too. No dataset or study is perfect. There are inherent limitations and uncertainties associated with data collection, sampling methods, measurement tools, and potential biases. A good study should transparently acknowledge these limitations and uncertainties to provide a more accurate representation of the findings.

Above all else, practice critical thinking and consider alternative interpretations. Rather than accepting information at face value, actively seek alternative explanations and assess the credibility of each perspective. This approach guards against falling victim to cognitive biases or accepting misleading information. Embracing skepticism doesn't mean dismissing evidence outright; instead, it involves a thoughtful analysis of all available information to arrive at the most plausible and evidence-based conclusion. Trust your gut - if something feels too good to be true (or too bad to be true), look into it further. Intuition can be a powerful tool, and a gut feeling can sometimes alert us to potential issues or opportunities that require further investigation.

Communicate and Share Data Responsibly

If you’re also in a position where you’re creating and/or sharing data, ensure that you clearly state the purpose and scope of your analysis. If you’re sharing someone else’s data, ensure that you’re providing context, or at least a link to the context. When you’re responsible for interpreting data or using someone else’s study for writing purposes, emphasize your own understanding before proceeding.

Above all else, use appropriate language and visuals to convey your message accurately. Using appropriate language and visuals goes hand in hand with promoting critical thinking and fact-checking. Encourage others to question information, evaluate sources critically, and verify claims before accepting and sharing them. Our digital spaces already have enough misinformation. It’s up to you to help with the accuracy of the internet. By valuing accuracy and truthfulness, we can collectively combat the spread of misinformation and foster a digital world where reliable information prevails.

–

By equipping ourselves with the knowledge and strategies above, we can avoid misinterpreting data and become more data-literate as non-scientists. Understanding the basics of data, cultivating a critical mindset, and applying practical analysis techniques enable us to navigate the data landscape confidently. Effective communication of data findings empowers us to contribute to a more data-literate society.

Data literacy is a continuous journey of learning and exploration, one that I’m still on too. With the skills acquired, we have the power to make informed decisions and actively participate in shaping a data-driven world. Let's embrace the opportunities presented by data interpretation and make a meaningful impact in our lives and beyond!