When complete decisions are a bad idea
Updated: Jul 25
Academic. Theoretical. Anecdotal. These are the kinds of things people say about business decision research – and they don’t mean it as a complement. Research about how people make decisions is often seen as “interesting but useless.” It passes the time on an airplane but doesn’t lead to better decisions as a practical matter.
Still, some insights can have a big impact on whether your decisions turn out right or wrong. One helpful nugget is the distinction between completeness and accuracy. The conventional wisdom is that people make better decisions if they do a more complete analysis (i.e. consider everything before making a decision). This is the school of completeness.
Another school focuses on accuracy. Predictions. In the case of accuracy, it’s not about how many things you consider but how few you consider, which few you consider and whether you end up right or wrong. It’s about weeding out bad criteria rather than gathering up everything.
Completeness is inclusive, accuracy is exclusive.
While both are important, businesspeople deal in the future. We have decisions to make and we live with the consequences. So in deciding what criteria to use for a big decision, it’s not ultimately about getting more data for the sake of data. It’s about finding the bare right data that can be used for an accurate prediction. Don’t cast a wide net. Rather, find the sharpest harpoon. Realize you’re in the land of prediction.
Some readers are thinking “no duh, of course you want to use the right data.” The alternative would be to intentionally use the “wrong” data. Yet I’m making a stronger claim – picking a side in the epic battle between completeness and accuracy. When business decisions are afoot, I’d rather be accurate. Leave completeness to the historians.
One way to visualize complete and accurate decision making is illustrated below. With the completeness of an analysis on the vertical access and its predictive accuracy on the horizontal, its worth conceding that the ideal falls in the upper right corner. The highest aspiration is to be both complete and accurate. We’ll call these decisions “correct,” in that they’re the most likely to work out.
While correctness is the pantheon of analytics, the unfortunate reality is that most of us lack the time and money to analyze every decision to its core. So as a practical matter, the next best thing is to be in the bottom right corner. These analyses use a small handful of inputs but predict right on target anyway. They may not consider everything in the universe (and can therefore be risky), but as a practical matter they can greatly improve an organization’s decision making. We’ll call these analyses “useful.”
The upper left corner is for “descriptive” analyses. Third on the pecking order, these aren’t really decision tools at all. They’re just a bunch of untested criteria; considerations. Things to think about. It’s a collection of candidate information vying for graduation into usefulness.
The bottom left corner holds the dregs. The “unreliable.” These analyses aren’t well thought out or accurate in a meaningful way. They’re a wild guess. Avoid advice from this corner no matter who’s giving it to you, what their credentials are or how intuitive they seem at first glance. Intuition is the last refuge to which a scoundrel clings.
A common mistake made by businesspeople, academics and overzealous authors is to confuse a descriptive analysis with a useful or correct one. When people do this they often say things like “we looked at a ton of data and felt it cumulatively led us to decide X over Y” while waving their hands in the air. It’s no surprise that such decisions tend to have random accuracy.
We all know even the best decisions can be ruined by unforeseeable change or bad luck. Life’s unpredictable, but that’s not to say we can’t improve our odds of making better decisions. We need only acknowledge that:
decisions are predictions in disguise;
predictions are about accuracy; and
accuracy is about finding the few right things to focus on at the exclusion of everything else.
This insight can make a huge difference in how businesspeople spend their time and money, what questions they ask and how they approach big decisions. If you’re a marketing manger, don’t spend your time analyzing everything customers tell you – spend it figuring out what’s predictive of their buying behavior. If you’re CEO, don’t ask your team for a massively complete analysis, ask them what few indicators are predictive of the outcome you’re looking for (if they don’t know, be concerned!). If you’re a consultant, stop bombarding your clients with 200 page presentations and hedging your bets with “on the one hand… on the other.” If you’re an investor, the implications of these concepts should be self evident.
Internalizing the difference between completeness and accuracy can lead to huge gains in operational effectiveness. It can also create significant improvements in decision quality and accuracy. That’s what I predict.