Why is it so difficult to measure a great idea?

Summary; For the past several years, I have been at the forefront of a movement called the science of implementation, or the science of scaling. In this work, we are trying to understand why some products, companies, and social programs thrive, while others go out of business.

When a seemingly promising idea loses its usefulness or profit, we call it a "voltage drop". On a scale, these failures are never the same.

Over the past 25 years, as a behavioral economist, advisor to large and small companies, and a former White House economic adviser, I have identified five reasons I call "significant symptoms" of voltage drops.

1. False positive

This happens when you interpret evidence or a piece of data as evidence that something is true, when in fact it is not - for example, as we have found in the false results of the Covid test. Have seen together For scaling, a false positive is a false sign that there is voltage in an idea when it really doesn't.

Sometimes false positives occur due to statistical errors, such as the well-known drug abuse prevention program D.A.R.E. After an independent study showed promising short-term results, the program began receiving funding from the US Department of Justice. However, there were several issues with the study: it excluded drugs such as alcohol and marijuana, focusing on tobacco. It was based on a small sample size; And later studies and even meta-analyzes could not replicate the results.

In other cases, false positive results are deliberately false. Think of Elizabeth Holmes and Theranos' alleged blood testing technology, which did not exist.

Whenever possible you should have all three of these components in place for launch to maximize profits. In undercover research companies, employees should be encouraged to pay financial rewards that encourage them to question the results.

2. Biased representation of the population

Once you've reliably demonstrated the effectiveness of the effort you hope to measure, the next step is to answer the question, "To what extent will the idea work?"

All ventures need to understand their potential audience. The first way to do this is to make sure that your test samples on a small scale reflect a large population. Otherwise, you'll be like McDonald's, who suffered from electoral bias when he launched the failed Arch Deluxe. Focus group participants liked the new product, but they did not represent the majority of Americans who just wanted to keep eating their Big Max.

To eliminate such prejudices, make sure your initial adopters are a random sampler. You should also make sure that your survey respondents have the appropriate privileges to tell you the truth. A Focus Group partner who says they would buy a product if it was introduced could only say, "I'd like to consider this product in the future, in contrast to the future." I will buy the product. "

3. Non-negotiable that cannot grow or be copied

To keep an idea or enterprise strong on a scale, you need to know if your "non-negotiable" - the motives for your success - can be mimicked on a scale. In other words, is your secret sauce "chef" or "ingredient"? Because people don't measure well (ie, they can't be cloned), talent-based projects often don't. You can't afford all the capabilities you need as you grow, so you hire low-performing actors and the quality is greatly damaged - a brutal voltage drop.

But, this important sign is more than just people. As you measure, regulatory constraints, resource constraints, sincere concerns, and many other issues can arise. Finally, we need to bring these scaling barriers back into the Petri dish and make sure Idea works with them.

4. Negative spillover

Spill over effect is the unintended effect that one event or consequence may have on another event or consequence. A good example is when a city opens a new factory, and the air pollution it produces affects the health of nearby residents.

As you scale, the chances of spoilers increase dramatically. The effects of general equilibrium, or natural market changes, are a major factor. I saw it for myself when I was the chief economist at Uber. A coupon that led to more rides in an area of Seattle when we spread it across the city as prices rose, and consumers found cheaper ways to get around that night.

There are also positive spreaders, such as network effects that make social media platforms more valuable as more and more people join them. When designing your idea early on, you should evaluate the negative spread and look for engineering opportunities and take advantage of the positive.

5. Cost net

To measure success, you need to determine not only how many people like your idea, but also what they are willing to pay for it and, more importantly, How much will it cost to supply?

When designing your enterprise, you have to account for two types of costs: default costs, such as a one-time investment in research and development to create a new product or service, and your ongoing operating costs. ۔ Initial costs may be reimbursed, but operating can bleed and cause a voltage drop, as with the innovative scientific wellness startup Arivale, which is a preventative health care provider. Ready to change care, only to go bankrupt in a few years, because it was not possible. Don't look for viable price points for its services.

One of the strategies to avoid the scaling cost trap is to make sure you take advantage of economies of scale. Elon Musk's expertise excels in all his projects. Ever since it helped transform the world of online banking on PayPal, it has promoted every major innovation in large-scale economies. Consider Tesla. Its great success can be traced back to the economies of its two main components: batteries and solar power generating cells, both of which can be manufactured significantly cheaper in large numbers. In addition, everything at Tesla is designed to enhance the performance of the "machine building machine", or what Musk affectionately calls his "Alien Dreadnought" - a state-of-the-art, fully automated production facility.

Another strategy is to build models that do not rely on advanced capabilities. As you measure, finding and paying high performers will be prohibited. The solution is to develop products that can give consumers their full value even if the average performer is delivering it.

As I speak on this subject, I would like to quote from the famous opening line of Leo Tolstoy's novel Anna Carina: "Happy families are all the same. Every unhappy family is unhappy in its own way. Expandable ideas are all the same. Every non-expandable idea is expandable in its own way. The difference with scaling is that you only have five major obstacles. And once you figure them out and avoid them. , So you can scale your idea for maximum voltage.