During World War II, the Allies mapped bullet holes in bombers that had been hit by enemy fire to reinforce heavily damaged areas and reduce bomber losses. They initially thought the areas with the most red dots (or bullet holes) should be strengthened since they had been the worst affected. In theory, it was a logical deduction. After all, these were the worst affected areas. But Abraham Wald, a mathematician, came to a different conclusion. He realized that the red dots only represented the areas where the planes could take damage and still make it back safely, while the areas without red dots were the ones that needed reinforcement because they were the ones that would cause the plane to crash.
This phenomenon is known as "survivorship bias," which occurs when we focus on the things that survived rather than those that didn't.
The survivorship bias is applicable in many areas of life, including healthcare, where websites may publish claims of miraculous cures but fail to mention the severe adverse outcomes experienced by others who took the treatment.
Survivorship bias can be a dangerous trap for businesses looking to replicate the success of their competitors. Often, companies assume that because a particular strategy worked for one successful company, it will also work for them. However, the strategy may be ineffective or even detrimental without considering the broader context of the company's history, structure, and market position.
A policy of radical transparency that includes direct and harsh public feedback may work for Netflix(1), and firing the 10% most underperforming staff might have been adequate at General Electric(2). But before adopting any of these policies, one should be wary of only looking at their use in successful companies.
The survivorship bias is not limited to the business world. It can also be seen in our personal lives, where we often focus on successful people and try to emulate their habits and actions without considering the broader context of their lives. We like to select winners who outperformed the rest, try to find their bright spots, and come to conclusions based on their personality and actions without looking more broadly at the whole dataset, including those with similar characteristics that failed to perform as well.
The New York Times(3) reported a new trend of young people choosing not to attend college. Which aspiring entrepreneur isn't looking at Bill Gates, Michael Dell, Larry Ellison, or Sir Richard Branson? They all have in common that they achieved great success without attending or finishing college, and this went not without notice. It is easy to believe that "if they can do it, I can do it as well."
However, when looking at a larger sample of those who don't attend college, a very different picture emerges rather than just the successful examples mentioned before. A UK study by the Department for Education shows that 88% of graduates were employed versus 72% of non-graduates.The average annual salary for graduates we $43.000 against $30.000 for non-graduates (4). Another recent study(5) shows how many of the wealthiest and most influential people graduated from college and elite schools (defined as the Ivy League schools plus many of the top U.S. universities). The study analyzed 11,745 U.S. business leaders, multi-millionaires, and billionaires. Although attending college isn't necessarily the path to becoming rich, looking at the whole picture rather than just the 'survivors' makes it clear that it does help.
The danger in basing your understanding of the world on those who have 'beaten the odds' or have succeeded in taking 'big risks' becomes evident if you carefully consider the logic of those phrases. Such people must be unrepresentative of the bigger picture, and thus one should be cautious of imitating them. After all, if everyone was succeeding by taking a significant risk, these risks can't have been that big, nor the odds of succeeding that formidable.
We may think that we can achieve success by copying particular characteristics. It is wise to remember that success can sometimes come down to something as unpredictable as a bit of luck.
In Summary
Survivorship bias is a common trap that affects businesses and individuals personally. We tend to focus on successful people and try to emulate their habits and actions without considering the broader context of their lives or the experiences of those who did not achieve success. As demonstrated by the example of the Allied bombers and the studies on college attendance, survivorship bias can lead to misguided assumptions and poor decision-making.
Success is not solely a result of individual characteristics but broader contextual factors and, sometimes, even luck. By broadening our perspective and considering a range of successful and unsuccessful examples, we can avoid this trap and make more informed decisions. We should be wary of focusing solely on the survivors and winners and instead consider the whole dataset to gain a more accurate and nuanced understanding of what it takes to succeed.
Notes
(1) Netflix reportedly encourage execs to widely explain why they fire people.
(2) Why GE had to kill its annual performance review after more than three decades
(3) New YorkTimes, Saying No to College, Nov. 30, 2012
(4) UK Graduate LabourMarket Statistics 2018, published 25 April 2019
(5) Jonathan Wai & Heiner Rindermann. What goes into high educational and occupational achievement? Education, brains, hard work, networks, and other factors. Published online 14 Mar 2017
(6) The Conversation: Joantahn Was & Heiner Rindermann