Poor Charts & Data to Blame for Shuttle Disasters?

On January 28, 1986, tragedy struck. I remember that day clearly. On board the Space Shuttle Challenger was the first school teacher destined to go into space. Seventy six seconds after liftoff, speeding at over 2,000 km/h, the shuttle disintegrated, killing all seven crew members. During the accident investigation, it became apparent that poorly presented PowerPoint slides and graphs, created by NASA engineers before the launch, possibly contributed to the shuttle’s demise. According to one expert, clear presentation of that data would have revealed a risk that frigid morning and persuaded NASA not to launch.

Florence Nightingale, photo credit H Lenthall, Wikipedia
Florence Nightingale (Source: Wikipedia)

The concept of graphical representation – solving communication problems through a visual language – goes back to the 14th century, even before Galileo. Many centuries later in the early 1800s, Florence Nightingale (1820 – 1910), known as The Lady with the Lamp, was one of the first to use the idea of a pie chart to create a new way to see the world. Named after the city of her birth in Italy, being hailed as the mother of modern nursing is only one of her life’s greatest achievements. Unknown to many, she excelled in mathematics and statistics from a young age.

Nightingale’s life story is remarkable for a woman growing up in the Victorian era. Even her parents encouraged her to study more “appropriate subjects”. She was first tutored in mathematics by her father and later studied the classics, the Bible and politics. Although she did not invent the particular pie chart she became famous for, her polar area diagram dramatised the needless deaths of soldiers in the Crimean War (1854 – 56) – not from their wounds, but from unsanitary conditions in military hospitals. Her charts revolutionised hospitals across the world, proving that good visual design can have momentous consequences and making her one of the most influential women of the 19th century.

Nightingale polar area diagram
Nightingale’s polar area diagrams she created during the Crimean War (Source: Wikipedia)

A modern-day statistician and information designer who has taken on similar challenges to Nightingale – for example investigating the London cholera epidemic of 1854 – is the man the New York Times calls The da Vinci of data. Edward Tufte, nicknamed E.T. by his peers, is a Professor at Yale University where he’s taught courses in statistical evidence, information and interface design. He’s written several seminal books including The Visual Design of Quantitative Information and Envisioning Information.

Visual representations of evidence should be governed by principles of reasoning about quantitative evidence. Clear and precise seeing becomes as one with clear and precise thinking.

–Edward Tufte, in The Feynman-Tufte Principle

As soon as NASA made public Challenger’s accident investigation data, Tufte – a master at making sense of data – studied the evidence and reasoned that the 13 charts prepared by engineers failed to reveal the risk that the shuttle faced. He created a single scatterplot chart from NASA’s data that would have been so convincing, according to the professor, that the shuttle would not have been launched and the astronauts would not have died.

NASA o-ring charts
One of NASA’s thirteen charts with ‘cutesy rocket icons’ (Source: Visual Function)
Edward Tufte scatterplot chart
Tufte’s single scatterplot chart (Source: Visual Function)
Fire emerging from right solid rocket booster
After liftoff, fire emerged from Challenger’s right solid rocket booster (Source: Wikipedia)

The prevailing theory as to what caused the shuttle’s disintegration was the failing of a rubber O-ring in its right solid rocket booster at liftoff. The orbiter’s two booster rockets, jettisoned after their fuel has been exhausted, are constructed with several joints; O-rings seal them so that high-pressure super-heated gas from burning solid propellant, reaching more than 3,000°C, can’t escape. On Space Shuttle mission STS-51-L, an O-ring failed, causing pressurised hot gas to breach the seal and engulf the orbiter in a fiery shroud. What Tufte was able to determine from the data, and that engineers knew, was that O-ring failure rates rose as temperature decreased; that Tuesday morning was a particularly cold launch, colder than any previous flight as Florida was gripped in a “100-year cold”. However, both the engineers and Tufte had too little data to go on.

The Rogers Commission – created by a Presidential Commission charged to investigate the shuttle disaster – included members like Richard Feynman (1965 Nobel Prize winner), Sally Ride (first American woman in space) and Chuck Yeager (first person to break the sound barrier). It concluded that it was an accident rooted in history. Both NASA and the Morton Thioklo managers (the booster rocket designers and manufacturers) were aware of the lack of information and knew that the design was flawed; NASA, despite knowing this, decided to continue the shuttle flights, judging them not to be severe enough to warrant the long delay required for a redesign. Ironically, the investigation and safety improvements following the Challenger accident caused a 32-month interruption in shuttle launches.

A paper in Science and Engineering Ethics (Vol 8, 2002) examined the role played by engineers (with reference to Tufte’s analysis) and highlighted the fact that they were aware of the problem and knew the design was flawed; O-rings became less resilient with colder temperatures and might not seal at all. Tufte assumed engineers had a complete set of data, which they didn’t (but had pleaded for), and consequently the authors point out that his eloquent scatterplot chart was incorrect. The paper concludes that engineers did what they were professionally and ethically required to do, going against Tufte’s assertion that their behaviour was unethical and guilty of an “overriding intellectual failure”. In this case, Tufte was not able to convincingly show that effective presentation could have prevented a disaster.

In the Columbia disaster 17 years later – when a piece of foam the size of a small briefcase broke off from the main propellant tank on liftoff and struck the leading edge of the left wing, causing the shuttle to disintegrate on re-entry – Tufte again blamed PowerPoint. During the critical period while Columbia was damaged but still in operation orbiting Earth, Boeing engineers presented their findings to senior NASA officials using PowerPoint slides. Tragically, after hearing the team’s report, NASA decided to go ahead with the re-entry, a disaster which accorrding to Tufte could have been avoided if engineers had not obstructed clear communication by supplementing supporting engineering data and analyses for poor PowerPoint slides.

If Tufte’s methods of clear thinking had been taken more seriously, history could perhaps have been rewritten.

Previously published

This article was originally published in Canopus, February 2011 (PDF)

Leave a Reply

Your email address will not be published. Required fields are marked *