Displays of Evidence for Making Decisions – Pt. II

On the 28th of January 1986 the space shuttle Challenger exploded and seven astronauts died due to the erosion of two O-rings. These rings lost resiliency because the launch happened on a very cold day.

untitled4

Launch permit

On the previous day, after seeing the weather forecast for launch day, the space shuttle engineers, responsible for its construction, advised the responsible agency, NASA, to postpone the event.

These engineers had several indicators that the launch could go wrong: history of O-ring deterioration on previous launches, the physics of resiliency, and other experimental data. All this information was faxed to NASA in 13 tables.

Although it was the only cancelling recommendation in twelve years,  NASA officials were quite surprised, they pointed several flaws on the presented tables and suggested a reconsideration.

And it was reconsidered.

And in the next morning, the Challenger space shuttle exploded 73 seconds after launch, due to the low temperatures affecting the O-rings.

Challenger explosion, 73 seconds after ignition

Tufte enumerates several mistakes on those 13 tables. Some had no ID and it was hard to track the responsability of who made the observation, sometimes the same shuttle had three different names. We can see some examples on the images below. They are note very clear about the already existent erosion on the O-rings under low temperatures.

untitled3

Some of the information and tables sent to NASA the previous day, suggestiong the cancellation of the Challenger launch

At a given time, NASA officials and engineers were focusing not on the erosion data but on the blow-by data. They’ve noticed that blow-by happened on the day with the highest temperature, just like it happened on the day with the lowest. The blow-by data were irrelevant in this case, and they’ve changed the focus of attention that should’ve been given to other crticial elements.

These 13 tables didn’t manage to avoid launch, but who made them was right: They were thinking casually, but they were not illustrating casually.

Tufte gathered the information in the tables and in other reports sent afterwards, and organized them as a function of temperatures, like so:

untitled7

Information gathered by Tufte and organized as a function of temperature

This way we can clearly see that there are more erosionn cases when temperatures were low.

In this matrix, also made by Tufte, this relation is even more obvious: the forecasted temperature was completely out of the accumulated data of 24 previous launches. untitled8

The same information presented on a matrix

After the incident, presidential commissions and investigation were convened. Their illustrations made the same mistake: they lacked labels, the cause-effect relation is not clear, and they are out of order.

untitled5

untitled6

Illustrations made by the presidential commission that investigated the Challenger accident. 

See what would happen if the information was ordered by temperature instead of cronologically.

untitled9

The same information, organized by temperature

Would anyone dare to launch the Challenger shuttle if the data was presented differently?

Although we often hear that data speak for themselves, their voices can be soft and sly.

– Frederick Mosteller, Stephen E. Fienberg, and Robert E.K. Rourke, Beginning statistics with Data Analysis (Reading, Massachusetts, 1983), 234

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s