“Latent Errors” (Errors Waiting to Happen) in Systems (Reason, 1990)

• Systems more automated
• Systems more complex, dangerous
• Systems have more defenses against failure (good)
• Systems have become more “opaque” (bad)
• Users may not know what is happening
• Users do not understand what the system can do
• The “ironies of automation” (Bainbridge, 1987)
  • Designers’ errors contribute to subsequent errors/accidents
    
  • Designer who seeks to eliminate humans through automation still leaves the human operator “ to do the tasks which the designer cannot think how to automate”