Brad Hargreaves | Building Things

Brad Hargreaves on entrepreneurship, community and life

Accuracy and Precision

with 5 comments

In conversation, the terms accuracy and precision are used interchangeably. But they mean different things, and the difference can play a big role in the growth of a business. Before getting into early stage companies I spent most of my time in a science lab, which couldn’t have put me in a worse position to understand how accuracy and precision affect startups.

Courtesy Jekel 1996

In science, precision is valued above accuracy. In this case it’s called repeatability, and being able to run the same experiment multiple times with the same results is a good thing. After all, most fields of science expect results with 95% confidence, which means that your error rate can be no higher than one in twenty. So controlling for all possible variables and demonstrating repeatability are of utmost importance.

In startups, this kind of thinking will get you killed on two fronts. First, achieving 95% confidence is impossible in business. If you can collect enough data to be right 60% of the time you’ll get buildings named after you. You can’t possibly control for all variables; you’re lucky if you have the time and money to even understand what they are. This can be summarized with with the old business adage “it’s better to be generally correct than exactly wrong.”

Second, running a variety of experiments that yield different results is a positive thing. If all of your business experiments look similar and yield similar results, you haven’t learned very much, and you certainly haven’t explored the full set of possibilities. In all likelihood there is a better outcome elsewhere, open for a competitor to find and exploit. In other words, you don’t want to optimize toward a local maxima while missing a bigger opportunity.

Take marketing strategy, for instance. Good entrepreneurs usually try a number of diverse strategies — perhaps PPC, plus SEO, plus events or social media — to get a few data points around what works and what doesn’t. Thinking about this as a fractal and trying a few diverse strategies within each of these categories can pay dividends as well. While most of these experiments will likely fail, they can provide multiple starting points from which to drill down and test further, or provide guideposts to the right answer.

So while the precision of experiments is not so critical, accuracy is key. With all your experiments, you want to be close enough within range to triangulate the right answers through experiments. While precision without accuracy is dangerous, being neither precise nor accurate is useless. Trying five wildly different social media engagement strategies for a beta product may yield a false negative if social media isn’t the right acquisition channel; picking a wider variety of tactics may have generated more interesting results.

Business isn’t science, but you can be scientific about it. Having the right experimental framework can go a long way to saving time and money.

Written by Brad Hargreaves

May 13th, 2012 at 12:38 pm

Posted in Uncategorized

Tagged with , ,

  • https://twitter.com/#!/sergeynazarov Sergey Nazarov

    Thanks for the insightful post Brad. It seems to me that it varies where you want to be on the spectrum of accuracy and precision depending on company stage. 

    Initially B as you try to see what the market signal will be either in terms of marketing channels, customer acquisition, user lifetime value, product market fit, etc… and then eventually (hopefully), you will reach C (except in the bullseye), which is where VCs have traditionally invested/scaled companies up until recently. 

    Viewing VC investing and entrepreneurship through this interesting lens; it seems that since the cost of business has gone down and because small teams with product can get to a large market faster you end up seeing many early stage companies/investments that are really just D, even if they don’t know it yet . This seems to be what happens when certain consumer internet products that do well in the smaller tech ecosystem are not actually attractive to the larger market and should therefore not prematurely scale/take money believing themselves to be at C and in the bullseye. 

  • Pingback: Accuracy and Precision | Bradley Stephenson

  • Pingback: Accuracy and Precision | Plans and Strategies

  • http://dissertations.biz/ phd dissertations

    You know, i can’t say that I fully agree with you, but it is a very curious theory indeed

  • Bctyner

    I disagree that science values precision over accuracy. Inaccuracy is like trying to measure height using a weight scale. Of course you will get the same value over and over (called reliability), but weight is an invalid measurement of height. 

    Reliability and validity have equal value in science: precision/reliability shows your results weren’t a fluke, accuracy/validity shows you are actually measuring what you intend to. 

    Your argument that accuracy is more important in business is interesting, though. Without scientific controls, I imagine a broad, scatter-gun approach is better because it increases your chance of hitting something than if all of your ventures land in the same market. So in a sense loose precision is a way to diversify your investments, but loose accuracy is just making bad investments.