The Downsides of Data Driven Decision Making

In life, and especially in business it is quite fashionable these days to hear people claim that they make "data driven decisions." Making data driven decision sounds like a very smart thing to do. Certainly much smarter than its regular language translation, "using the facts to determine a course of action" which sounds awfully obvious and boring. Definitely not something one would see published in the Harvard Business Review. Despite its obviousness it would seem hard to argue with the sagacity of making "data driven decisions", after all who would not want to use all of the information at one's disposal (data) in deciding which to select of the many possible alternative options that are available (decision making) for doing whatever it is one is trying to do? Usually that thing one is trying to do is make money or some derivative thereof. Nothing gets me more fired up for an argument than a thing which is hard to argue with. Therefore, below I present to you two arguments against data driven decision making. Of course I am in general a big supporter of data driven decision making and in no way mean to suggest that it is not (most of the time) the smartest way to go about making decisions in just about any aspect of one's life or career. However, that said, it is by no means guaranteed to be the best way to make decisions or even the smartest way to make decisions in every case. As you will see if you continue on, data driven decision making has at least two highly relevant downsides that can make it the less than optimal mechanism for deciding things. Even if less than optimal in some specific instances, it is still likely never a bad way to make a decision, it just won't guarantee that you always come out on top. Nothing ever does though does it?

Data driven decisions are slow decisions

It takes time to gather data, analyze it, prepare reports, give presentations, etc. Basically to do all the stuff that needs to be done in order to lay the groundwork for making data driven decisions. This is fine, and it should be that way. However, in a competitive marketspace, their will always be many players who are not making "data driven decisions" and are instead wild ass guessing or intuiting their decisions. Occasionally, more often than you might think in a crowded marketspace, some of these people will make the correct decisions just by random chance. They will make these decisions well before you have time to do the slow work of number crunching required of data driven decisions. By the time you have completed that work, and are ready to make your decision, it may be too late, you have lost first mover advantage or any other number of bad effects that happen when you are slow to the game in business. While you were in the library doing research they were out on the field, doing stuff, real stuff. The sorts of real stuff required to run a modern business. Eventually the world will catch up to those fakers who didn't make data driven decisions and they will fail is a common refrain one hears from data driven decision evangelists. In truth they might be correct, but until they do the DDDe's arirt and wasting time in the library when you should be out on the field of play

Data is generally imperfect and always incomplete

Any data driven decision one makes will only be as good as the quality of the data one is using as the input to that decision. Data, by its very nature, is only as good (useful) as the method's used to collect it, and is rarely 100% reliable or complete. Therefore any decision one makes based on said data will always be biased in whatever way the data is biased, and, incomplete data will make for imperfect decisions.

Deciding how to analyze the data is the biggest driver of the outcome of the data driven decision making process, and it is easily biased

Ironically it is a decision, which data analysis tool/package/approach shall be used to analyze the data upon which the decision will be made, that has the biggest impact on the outcome of any data driven decision process. Does one use a data driven decision process in deciding this? Is it just data driven decisions all the way down? Probably not. In fact, the approach used to analyze a given data set was probably picked for one of many very mundane reasons. For example, the person doing the data analysis is familiar/comfortable with it and has the software/skills/knowledge necessary to use and understand it. People analyze data the way they have always analyzed it in the past and change only very slowly with time. This is true even if the way they are analyzing a given data set is far from the most optimal way it could be analyzed. It is also very tempting and quite easy to 'pick and choose' a data analysis approach that is aligned with ones internal biases and is more likely to give a predetermined answer. Realistically however data driven decisions are usually very complicated decisions and no one data set or data analysis tool will be used when trying to reach the ultimate decision. This is a good thing but it brings us right back to the very first problem I described above, the slowness of data driven decisions.