Human Systems Integration Division @ NASA Ames

"Five human performance models differing in architecture, fidelity and prior application were applied to taxi navigation. Each model was assessed for its utility in generating and explaining a subset of the 12 navigation errors that were actually committed by airline pilots participating in a simulation. These errors are examples of the type of human error cited as causing 60 to 80% of aviation accidents. Efforts to improve the safety of the civil aviation system and reduce accident rates must address the underlying causes of human error and develop strategies for prevention and mitigation. Effective human modeling and simulation can flag error-prone designs before expensive prototypes are built or systems fielded. Human performance modeling tools can evaluate the performance implications of new design concepts early in the development process.

A formal review of human performance models revealed a variety of architectures and approaches, but few that had attempted to predict errors. Since many were developed and validated primarily with laboratory data, applying them to the dynamic, interactive complexities of aviation was a challenge. Each type emphasized certain aspects of human perception, cognition, and behavior over others and with varying fidelity, thus no one approach would be likely to capture all of the behaviors and failures inherent to the domain. To gain a better understanding of the capabilities and limitations of human performance models, five approaches were selected for development and application to a specific aviation problem: (1) ACT-R/PM (Atomic Components of Thought /Perceptual Motor) is an activation-based production system architecture of cognition with enhanced perceptual and motor processes; (2) Air MIDAS (Air Man-Machine Interface Design Analysis Tool) is an integrative model of a goal seeking human(s) immersed in a context rich environment; (3) A-SA (Attention and Situational Awareness Model) is a computational model of attention and level of situational awareness; (4) D-OMAR (Distributed Operator Model Architecture) is a general purpose, event-based simulation development environment; and (5) IMPRINT/ACT-R (Improved Performance Research Integration Tool/Atomic Components of Thought) is a procedural task network model linked with an activation-based production system architecture of cognition. All but one (A-SA model) was well-established with numerous prior applications and validation efforts.

Each modeling team constructed a dynamic representation of the taxi-navigation portion of a recently-conducted, high-fidelity, full-mission simulation. These deviations from the assigned course as pilots taxied after landing were seen as especially useful because the errors were clearly specified, well-documented and very realistic. In Figure 1, a screen capture from a DOMAR animated runtime display shows the movements of two conflicting aircraft on taxiways, along with model-generated pilot and ATC communications. Explanations derived from the models included instances of decision errors resulting fr poor situational awareness, crew coordination problems due to distraction and workload, and execution errors based on heuristic biases invoked by time pressure. Predictive capabilities were demonstrated by some of the models (e.g., new errors were generated that extended and slightly varied the original taxi conditions and events). The modeling results confirmed both the promise of human performance modeling and demonstrated the challenges. "