How to be Smart About Statistics

"There are three kinds of lies: lies, damned lies, and statistics." - Mark Twain.

"You can fool all of the people some of the time; you can fool some of the people all of the time, but you can't fool all of the people all the time. " Abraham Lincoln.

Every day, we hear and read statistics on any number of things, from how the economy is doing to the relative safety of our cities, and hearing these statistics make us feel more intelligent. We make decisions in our lives and careers based on what we hear. But statistics are always correct, nor do they always reveal what we're being told. So it's best not to immediately accept them without giving them additional thought. Here's how you can train yourself to be more discerning about the statistics you encounter.

Sample size

The first thing to keep in mind about statistics is where the numbers are coming from. What would you think if you heard the statistic that 80% of Americans smoke? Most likely you'd think that was high. 60 years ago that may very well have been true, but every since studies showed smoking causes cancer, emphysema, and other diseases, it's been in decline, especially in the last 30 years when smoking indoors was outlawed in most places across America. Given that, how could it be that 80% are now smoking?

Well, it could be that the survey was only done in one state, perhaps Mississippi, which has a far higher than average number of smokers. The number was then applied to the rest of the country, not accounting for differences in state, age, or anything else. On a recent trip I stopped at a gas station in rural Alabama and was surprised to see that not only were there people smoking inside, it was most of the people. If the poll was done just of those people, you could reach the 80% statistic, and the statistic would be true of that gas station, but not of the whole state, region, or country, because the sample size was too small.

Targeted numbers

It's plain to see that the United States and many other countries are sharply divided on politics, and it's common for people who have common opinions to live in the same area. In America, people in cities tend to be more progressive and in rural or suburban areas they tend to be more conservative, with some neighborhoods and cities being extreme versions of either view.

This presents a problem when you have polls of supposedly a lot of people, but they're only done in one area.

Two excellent examples are in Wisconsin. What if you heard three conflicting surveys: One that says 70% of registered voters say they would vote for President Biden in the 2024 election, while another says 80% would vote for Florida Governor Ron DeSantis. The third poll only has each of them at 50 or 55% Well, you can only vote for one candidate, so which poll is wrong?

Upon investigation, you find that each poll was of 1,000-1,200 people, which is good. Though it's a tiny fragment of the overall population of even just Wisconsin, which has about 5 million people, the average poll in the US is of about 1,000 people. However you find that first poll mostly questioned people in the city of Madison, which is one of the most liberal cities in America. The second poll talked to just as many people, but they were from towns throughout Waukesha County, which has the highest concentration of conservatives in the state. The third survey tried to find people from various areas, resulting in a much more even, less lopsided result, and really none of the results may show a true picture of what the majority of people think, or what election results would be.

Get the whole story

People in office want to talk about their successes so they can stay in office, or maybe get a better job, while their opponents talk about perceived failures. Often, the truth lies somewhere in the middle.

In one recent example, while discussing the state of the economy, the President cited job creation as a sign that the American economy is in great shape. However, nationally, the housing market is slowing down, spending per person is still down, and homelessness is up. What's the real story?

Well if you look at the study cited, the number of American jobs is increasing, but another study shows the number of average hours worked, is going down. Combined, the studies give you the real picture. People aren't working fewer hours because employers are so flush with cash that they can hire more people and have them all work 35 hours instead of 40 for the same pay. Rather, hourly workers are being scheduled for fewer hours, while employers are hiring more people to carry the load. Why? Benefits.

At most companies, if you work full-time, you're eligible for benefits such as insurance, vacation days, etc. One company's definition of full-time may differ from another, but generally if you're working an average of 30 hours per week each month, you're a full-time employee, and benefits are expensive. Thus an employer may choose to employ 20 people to work 20 hours per week rather than 10 people to work 40 hours per week, because those 10 full-time people would all get benefits, increasing the cost per employee by as much as 29.8% according to recent estimates. From the employer's perspective, the work is still getting done, and they're saving money.

However, there are consequences of that decision. The 20 people working 20 hours a week might have the same hourly rate as if they worked 40 hours per week, but they're working fewer hours, probably not enough to pay their bills on their own. That means they'd have to have another source of income, which creates scheduling conflicts, fatigue, and other issues. It would also potentially create dissatisfaction with their position, increasing job turnover, ultimately creating an overstressed, short-staffed workplace. This is not the sunny situation the president described, nor is it likely what you'd hear or see when casually getting snippets of news.

Correlation is not causation

Sometimes things happen at the same time, but independently. This is the area of statistical analysis where scientists, scholars, and especially lawyers can really make a mess of things.

Let's use smoking as an example again. People smoked for a long time before the Surgeon General linked it with cancer and other problems in the 1960s, and issues like a smoker's cough or "decreased wind" were commonly known, but at the same time, it wasn't difficult to find older people who were still regular smokers. Even today, while there are fewer smokers in general, one can find older people who still actively smoke. Famously, legendary comedian George Burns continued to smoke 15 cigars a day until he died at age 100.

While suspicions mounted that smoking might cause health problems, counter-research could find other causes. Someone's cough was due to allergies, or environmental pollution, and increased blood pressure was due to bad diet, or a family history of high blood pressure. This is done over and over again whenever something is linked to something else. Researchers paid to find a link between a product or activity will find that link, and researches paid to deny that link will say it's something else. What matters again are the sheer numbers.

To illustrate this to most hilarious effect, and at the same time, tie all these concepts together, I'll give you an example of a friend of mine, now deceased. He called himself Montana Mike, who was a self-described cowboy and scholar who spent most of his life in Montana.

Mike was a bit of a rogue, outlaw and ladies' man who ultimately left Montana because the woman he was dating at the time tried to shoot him while he was in the bathroom. She missed him, but hit the toilet, and he grabbed his stuff and got out of Montana. A lifelong smoker, he ultimately died of a heart attack after he refused to get a recommended pacemaker.

Knowing just these facts, you could make the following extrapolations, write articles about them, and sound convincing but still be totally wrong unless you had sufficient other data from reflecting the relative health and behavior of older men from Montana:

Case studies show the bathroom is the most likely location for gun violence in the home.

Case studies show men aged 60-70 are more likely to die of a heart attack than gun violence, cancer, or other causes.

Case studies show men aged 60-70 are more likely to be attacked by significant others in the home than to be attacked by strangers.

Case studies show alarming increase in bathroom gun attacks from users not burning scented candles.

New statistic shows Montana population decreasing as number of bathroom attacks increase 100%.