In studying cases of engineering failure (and disaster), we often wonder what could have been done to avoid the problem in the first place. Questions often arise, such as “Didn’t anyone notice that …” or “How could this have been designed without anyone realizing that it was a disaster waiting to happen?” As professionals, engineers are often called upon to assess, inspect, certify, evaluate, etc. complex structures, systems, devices and software, to help both designers and those charged with operations and maintenance to catch problems before they happen. These engineers are often identified as “experts” in a particular area, and their judgement is highly valued. But this brings up other important considerations. For example, exactly what makes a person an “expert”, and how can engineering judgement best be used to help avoid failure?
In considering these issues from an educational perspective, I recently encountered a very interesting paper which is available online: “Engineering judgement in reliability and safety and its limits: what can we learn from research in psychology” by Lorenzo Strigini of the Center for Software Reliability, at City University Northampton Square, London. The paper can be downloaded at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.3931&rep=rep1&type=pdf. While the paper is from 1996, it provides an excellent lesson for engineers today, starting from a psychological perspective.
The author considers such issues as the ‘presumption’ of expertise, the role of experience and background knowledge in evaluating reliability, and the value of an informal judgement process versus a structured methodology. The latter is especially important when the “expert” is asked for their judgement in a situation where they encounter a possible fault with which they have little personal experience.
A number of psychological factors which are critically important in considering the nature of engineering failures (and how they managed to occur despite supposedly thorough analysis and inspection) are reviewed in this paper. Overconfidence is identified as a leading contributor to failure, as is the existence of various biases in the judgement of engineers. Examples of overconfidence are abundant — for example, the space shuttle Challenger disaster, on which much has been written.
An interesting psychological bias described by the author is “hindsight” bias — as in “hindsight is 20/20”. But hindsight can skew our impression of how a failure occurred. In the words of the author:
“When reviewing a sequence of events and decisions which ended in failure, we build a theory that predicts what we already know to have been the final outcome; then, the decisions which preceded it appear to have been wrong: we no longer recognise the dearth of information, or the ambiguity of the information available, at the time decisions were made.” This is an important concept to keep in mind, especially for those of us using case studies to teach about learning from failure — while in hindsight, the causes of failure may seem obvious, we need to put ourselves in the shoes of the engineers and others who designed the system, operated it, or were present when the disaster occurred. Doing so may provide us with new lessons for how to better avoid problems in the future.
The conclusions of the paper describe a number of ways to ensure that engineering judgement and the opinions of experts can best help us avoid failure — for example, using multiple experts (as a check) and using structured methods for analysis and failure prevention. Techniques such as Failure Modes, Effects and Criticality Analysis (FMECA) can be used to help remove some of the biases and overconfidence from the process, and are often taught in engineering design courses. I will add some web links to information on these methods, and their applications, to this blog.