Beware of survivorship bias when interpreting data

During World War II, researchers at the Center for Naval Analysis faced a critical problem. Many bombers were getting shot down on runs over Germany. The naval researchers knew they needed hard data to solve this problem and went to work. After each mission, the bullet holes and damage from each bomber was painstakingly reviewed and recorded. The researchers poured over the data looking for vulnerabilities.((Syed, Matthew. 2015. Black Box Thinking. New York: Penguin Random House. pp 33-37))

The data began to show a clear pattern (see picture). Most damage was to the wings and body of the plane.

Source: When data gives the wrong solution

So, you reinforce those areas and you’re done, right? That sounds like it would be the correct course of action, but it’s wrong.