Ah, That's Some Robust Theory You've got there!

Science communication is broken. And I don't just mean when scientists are communicating to the average person. Scientific papers are riddled with poor terminology that illustrate a severe lack of understanding of the mechanisms that underlie scientific inquiry.

At the heart of the issue is scicomm's reliance on terms relating to confirmation. In science, there is no such thing as confirmation, verification, or support. All science can do is tell us, at best, when we are wrong. And yet, throughout both science related news articles, and in research papers themselves, one can easily find such terms being used constantly.

Some Quick Philosophy of Science

I want to focus on scicomm, but to really understand the issue here, I suppose it makes sense to go over some basic philosophy of science. When scientific inquiry itself was being developed, there was a debate over whether science could confirm our suppositions, and if we could really be confident about our future predictions. This question is called the problem of induction.

The only solution to date has been to describe science as a system of falsification, where we take our proposed theory or current working theory, make predictions using it, and test it against observations. If our theory stops making predictions that come true, then the theory is inconsistent with the body of scientific observations.

Hypothesis testing using p-values, or Bayesian inference, are the primary methods that we use to determine whether or not a theory is inconsistent with a given body of observations. But even this process leads to provisional falsification, because a larger body of future observations may lead to the evaluation that the once falsified theory is indeed consistent with the body of scientific observations.

But we cannot say much at all if the working theory is consistent with observations. It doesn't mean that the theory is likely to be true. It doesn't even mean that we really can trust the predictions made using the theory. It's more that we have little choice. Still, we can at least try to work with the most robust theories that we can find. But what makes a theory robust?

Robustness of a Theory

We cannot say that a theory is true, confirmed, or even supported. We need a better word, and "robust" seems to work. The more robust a theory is, the more reasonable it is to rely on it, because we've exhausted other options.

What makes a theory robust? First, it can't be contradicted by the current observations. So a robust theory must be consistent with the available data. But what if there is very little data? It doesn't make sense to call a theory robust if we haven't really tested it. So we must have a significantly large body of data, and have exhausted all reasonable ways to potentially falsify the theory. And to gather such data, the theory must be able to make a wide range of predictions. Furthermore, the more competing theories that are also consistent with the working body of data, the less robust any of those theories are.

Finally, it makes sense to say that a theory is more robust if a large number of other theories rely on it. Evolutionary theory, for instance, is pivotal to psychology, medicine, and other areas of research. If the evolutionary theory of common descent were falsified, it would cripple numerous other theories and force us to rework our understanding of much of the world. Therefore, such theories are quite robust.

Summary of Terminology

Even though this article is short, there is a lot to unpack here. So to summarize:

1. A working theory is inconsistent with the body of scientific observations if the observations are unlikely, assuming that the theory is true.
2. A working theory is consistent with the body of scientific observations if the observations are likely, assuming that the theory is true.
3. A working theory is robust if it is consistent with the current body of scientific observations; the theory has been extensively tested against a large number of observations, to the point that further testing would be impractical or impossible; preferably the theory acts as a foundation for a number of other theories; and there are few competing theories that satisfy these requirements.

These terms are not widely accepted. I use them personally, and hope that other science communicators and researchers do the same. Unless a way around the problem of induction can be addressed, we must accept that science is a system of falsification only.

Further Reading

I do have some mathematics on a possible solution to the problem of induction. However, it is unlikely that it will produce valid results, and even if it does, it creates numerous other issues.