Spec Check
NASA defines the "root" cause of mishap as [a]long a chain of events leading to a mishap, the first causal action or failure to act that could have been controlled systematically either by policy/practice/procedure or individual adherence to policy/practice/procedure.
The root causes of these mishaps (incorrect units, invalid inputs, inverted G-switches) seem obvious in retrospect. How could anyone have possibly made those mistakes?
In addition to the root cause, the MIB Reports also identify a "contributing" cause as [a] factor, event or circumstance which led directly or indirectly to the dominant root cause, or which contributed to the severity of the mishap.
The MIB Reports list many contributing causes for each root cause, far more than I have room to discuss. Some contributing causes seem familiar, because they describe precautions every project, even ours, should include.
MCO had a software interface spec and a well-defined testing process. However, the MIB discovered that [t]he Software Interface Specification (SIS) was developed but not properly used in the small forces ground software development and testing. End-to-end testing ... did not appear to be accomplished. ...The interface control process and the verification of specific ground system interfaces was not completed or was completed with insufficient rigor.
The MPL MIB observed that, even though the landing sensor switch glitch was identified as a potential failure mode by...fault-tree analysis prior to EDL, the description of the software design and testing provided at that time by [Lockheed Martin Astronautics] did not leave any concerns in the mind of the [Mission Safety and Success Team]. ...Ultimately, it was discovered that the software did not behave in the manner intended...
The Genesis project's G-switch layout error survived far more intensive scrutiny than most of your designs:
• The design review process did not detect the design error;
• The verification process did not detect the design error; and
• The Red Team review process did not uncover the failure in the verification process.
Although those contributing causes seem to put the final responsibility on the worker bees either building the widgetry or verifying it, the MIB Reports go further than you might expect, back into familiar territory.
