The WWII plane armor problem is the classic example of survivorship bias: looking only at aircraft that returned and forgetting the ones that did not. Abraham Wald’s insight was that the most important evidence could be the missing evidence. If bullet holes were common in one area of returning planes, hits there may have been survivable rather than the best place to add armor.
The WWII aircraft problem and Abraham Wald’s answer
During World War II, analysts examined combat aircraft that made it back to base. The returning planes showed visible damage, with many bullet holes in areas such as the wings and fuselage. A natural recommendation was to reinforce the areas with the most holes.
Wald, a mathematician working with the Statistical Research Group, reversed the question. These planes were not a random sample of all planes hit in combat. They were a sample of planes that had been hit and still survived long enough to return.
That distinction changes the recommendation:
- Areas with many holes on returning planes were areas a plane could often take damage and survive.
- Areas with few holes, such as engines, may have been underrepresented because a hit there was more likely to stop a plane from returning.
- Therefore, armor should be considered for the apparently undamaged critical areas, not automatically for the most visibly damaged ones.
The lesson is not “always protect the places with no damage.” It is: first ask what process determined which cases you get to see.
For a broader introduction and more everyday examples, see Survivorship Bias Explained: Examples, Mistakes, and a Quick Quiz.
Why the obvious answer feels convincing
Human attention follows what is visible. Returning aircraft provided maps, photographs, and holes that could be counted. The aircraft that failed to return could not offer the same direct record.
That creates a distorted dataset:
| Question | What the returning planes can tell you | What they cannot tell you alone |
|---|---|---|
| Where did survivors get hit? | The location of survivable damage | The location of all combat hits |
| Which areas had few holes? | Those areas were rarely damaged among survivors | Whether they were rarely hit or fatally hit |
| Where should armor go? | Not enough by itself | Requires reasoning about the missing aircraft |
The error comes from treating “frequently observed among survivors” as equivalent to “common in the whole population.” It is not equivalent when survival affects observation.
This is closely related to the difference between valid deduction and evidence-based inference. If you want to sharpen that distinction, Deductive vs. Inductive Reasoning: Differences, Examples, and Practice Questions is a useful companion.
The core logic, step by step
Use this sequence whenever a dataset consists only of winners, survivors, customers, graduates, published studies, or successful products.
1. Identify the observed group
In the Wald example, the observed group is not “WWII aircraft.” It is “WWII aircraft that returned from combat.”
That wording matters. A sample can be large and carefully measured yet still be incomplete in a systematic way.
2. Name the missing group
The missing group is aircraft that were hit and did not return. Their damage patterns may differ sharply from the survivors’ patterns.
A good diagnostic question is: Who had to disappear from the data for these results to look this way?
3. Ask whether inclusion depends on the outcome
An aircraft appears in the inspection data only if it survives the relevant damage. Therefore, survival influences inclusion in the sample.
This is the hallmark of survivorship bias. The data-generating process filters cases before you analyze them.
4. Reinterpret the visible pattern
Many wing hits on returning planes do not show that wing hits are the greatest threat. They show that wing-hit planes often remained capable of returning.
Few engine hits on returning planes do not prove engines were rarely targeted. A plausible alternative is that engine hits were often catastrophic, leaving no returning aircraft to inspect.
A quick test for survivorship bias
When an argument uses examples of success, run this four-question check:
- What group is being counted?
- Which comparable cases are absent?
- Could the missing cases have different characteristics?
- Does the conclusion assume the observed group represents everyone?
If the answer to the fourth question is yes, the reasoning may be affected by survivorship bias.
For example, imagine someone says: “The founders of several major companies left university, so leaving university is a reliable route to business success.” The visible founders are easy to name. The far larger group of people who left university without building major companies is missing from the comparison.
The problem is not that the successful founders are irrelevant. The problem is that they are an incomplete basis for estimating the typical result.
Practice: can you spot the missing evidence?
Try each item before reading the explanation.
Practice 1: The armor recommendation
Premise: Returning aircraft have many holes in their wings but comparatively few in their engines.
Conclusion: The evidence suggests investigating engine protection rather than assuming wings need the highest armor priority.
Is the conclusion reasonable?
Yes. The conclusion recognizes that the sample includes survivors only. Few engine holes among returning aircraft may indicate that engine damage often prevented a return. Notice the cautious wording: the evidence suggests investigating. It does not claim that the pattern alone proves the final engineering decision.
Practice 2: The app-success list
Premise: A report profiles 50 still-operating mobile apps. Most of them changed their pricing model in their first year.
Conclusion: Most mobile apps launched in that period changed their pricing model in their first year.
What is the flaw?
The conclusion generalizes from surviving apps to all launched apps. Apps that shut down may have followed different pricing paths, so the report’s sample cannot establish what was typical of the full launch population.
The premises support a narrower conclusion: most of the still-operating apps in the report changed their pricing model.
Practice 3: The study strategy claim
Premise: An instructor interviews students who earned very high scores. Nearly all say they completed every practice set assigned.
Conclusion: Completing every assigned practice set guarantees a very high score.
What should you challenge first?
Challenge the selection of interviewees. Students who completed every set but did not earn very high scores are absent. The word “guarantees” also goes beyond the evidence, but survivorship bias is the central issue because the sample was selected by the successful outcome.
A stronger conclusion would be: completing assigned practice sets was common among the interviewed high scorers. That is informative, but it does not establish a guarantee.
How this appears in LSAT-style flaw reasoning
Survivorship bias often shows up as an argument that relies on a filtered group while speaking as if it represents an entire group. The wording may mention successful businesses, recovered patients, accepted applicants, bestselling authors, or completed projects.
The practical move is to restate the sample precisely:
> The evidence concerns cases that survived a selection process, not necessarily all cases that entered it.
Then compare that statement with the conclusion. If the conclusion reaches all cases, look for missing failures or non-selected cases.
This is one of several ways an argument can use evidence that is less representative than it appears. For a systematic method of testing premises against conclusions, read How to Approach LSAT Flaw Questions: A Step-by-Step Method.
What survivorship bias does not mean
Not every success story is biased. A success story can provide useful evidence about how one person or organization achieved a result. The mistake begins when that story is used to estimate how likely the same approach is to work overall without examining comparable failures.
Nor does survivorship bias mean that every low-count category is secretly dangerous. In the plane example, few observed engine hits could have more than one explanation. Wald’s contribution was to recognize that the visible data did not settle the question without accounting for the missing aircraft.
Good reasoning keeps both points in view: the survivor data matter, and the absent cases may change what the data mean.
FAQ
Was Abraham Wald’s WWII plane armor solution really about putting armor where there were no bullet holes?
Broadly, yes, with an important qualification. Wald’s reasoning was to focus on vulnerable areas that were underrepresented among planes that returned. The point was not a blanket rule to armor every unmarked area.
Why are the bullet holes on returning planes misleading?
They show where surviving planes were hit, not where all planes were hit. Damage patterns on aircraft that did not return may have been different.
Is survivorship bias the same as confirmation bias?
No. Confirmation bias involves favoring information that supports an existing belief. Survivorship bias involves drawing conclusions from a group selected by survival or success while overlooking the missing cases.
How can I recognize survivorship bias on a reasoning question?
Look for evidence drawn only from successful, surviving, selected, or visible cases. Then ask whether the conclusion assumes that group represents failures and non-selected cases too.
Conclusion
The WWII plane armor problem is memorable because it turns an intuitive conclusion upside down. The bullet holes on returning aircraft were evidence of where planes could often survive being hit; the missing planes carried the evidence that was hardest to observe. When a claim is built from visible winners or survivors, define the missing group before trusting the pattern.
Want to test whether you can spot the missing-planes mistake in fresh scenarios? Take the interactive survivorship bias quiz.