feary_SAP_research0103.html

Cognitive Usability Engineering and Analysis Research

The wide-spread application of Information Technology to automate tasks in our everyday lives has improved our overall productivity and increased our safety. Researchers, however, have identified a circumstance in which the application of these technologies has resulted in increased operator workload that has lead to reduced safety margins. This occurs when the automation induces operator error by requiring the operator to exceed the natural limits of human cognition.

More information about automation research conducted at NASA Ames can be found at: http://automation.arc.nasa.gov

The Cognitive Usability Engineering and Analysis Project is part of the Human Automation Interaction Element of Airspace Operations Systems Project of NASA's Airspace Systems program. In particular this research identifies the characteristics of training, procedures , and automation design that optimize the interaction between human and machine.

Two technologies have been developed through this research:

A method for analyzing automation to identify two characteristics of automation user-interfaces that are known to cause operator overload. Demonstrated on user-interface in modern cockpit. (1)

A web-based tutor for training complex skills that reduces time to competence and improves retention. Demonstrated training pilots to use a modern Autopilot.(2)

1. Cognitive Engineering/Usability Analysis

The wide-spread use of Information Technology to automate everday tasks has improved productivity and overall safety in industries such as transportation, healthcare, and manufacturing. Researchers have, however, identified products in which, under certain situations, the cognitive capacity of the operator is exceeded. This has resulted in increased operator workload, and has lead to degraded safety (in these situations).

A formal methodology has been developed to analyze information technology systems that automate complex tasks to identify a set of design characteristics known to cause increased cognitive workload. Unlike tradional methods that analyze a hypothetical model of the operator's behavior, this method analyzes the device software.

The method, developed under NASA funding, has been used to evaluate aircraft cockpit user-interfaces.

Background Papers:

Feary, M. and Sherry, L. Dynamic Storyboards: Faster Cheaper, Software Prototypes for Usability Evaluation. Conference on Human-Computer Interaction in Aeronautics, Toulouse, France, (29 September - 1 October).

Feary, M., Sherry, L., Polson, P., and Palmer, E.(2001). A Formal Method for Integrated System Design. 8th European Conference on Cognitive Science Approaches to Process Control (CSAPC ’01), Munich, Germany (24-26 September).

Sherry, L., Polson, P. and Feary, M. (2002) Designing User-Interfaces for the Cockpit: Five Common Design Errors, and How to Avoid Them. 2002 SAE World Aviation Congress, Phoenix, AZ (November 5 – 7).

Sherry, L., Polson, P., Feary, M. and Palmer, E. (2002) When Does the MCDU Interface Work Well? Paper presented at the the International Conference on Human-Computer Interaction in Aeronautics, Cambridge, MA (October 22-24).

Sherry, L., Feary, M., Polson, P., and Palmer, E. (2000). Formal Method for Identifying 2 Types of Automation Surprises. Honeywell Publication C69-5370-016, Phoenix, AZ, USA.

Sherry, L., Feary, M., Polson, P., Mumaw, R., and Palmer, E. (2001). Cognitive Engineering Analysis of the Vertical Navigation (VNAV) Function. NASA/TM-2001-210915. Ames Research Center, Moffett Field, CA, USA.

Feary, M., McCrobie, D., Alkin, M., Sherry, L., Polson, P., Palmer, E. (1998). Aiding Vertical Guidance Understanding. NASA/TM-1998-112217. Ames Research Center, Moffett Field, CA, USA.

Sherry, L., Feary, M., Polson, P., and Palmer, E.(2001). What's It Doing Now? Taking the Covers Off Autopilot Behavior.

Sherry, L., Feary, M., Polson, P., and Palmer, E.(2001).Why Won't It Take That? MCP Control Device Invokes More Than One Behavior.

Feary, M., and Barshi, I. (1999) “Why is it Doing That?” Perspectives of the FMS. Paper presented at the Tenth International Symposium on Aviation Psychology, Columbus, OH (May 3 -May 7).

2. Cognitive Tutors

Advances in cognitive science, coupled with the capability of the web, provide the basis for improvements in training complex skills. Tests demonstrate faster skill aquisition (1/3 faster) and better retention.

The Autopilot Tutor builds pilot skills to operate a modern autopilot. The tutor and the workbook identify the core knowledge concepts and then develop pilot skills by guided drill-and-practice.

As the pilot learns-by-doing, the tutor reinforces the correct behavior, and immediately corrects improper behavior.

You can examine the Autopilot Tutor Workbook and then perform the exercises on the Autopilot Tutor Web Tool. (Note: This may not work outside of NASA for now - sorry)

Background Papers:

Sherry, L., Feary, M., Polson, P., and Palmer, E. (2001). AutoPilot Tutor: Building and Maintaining AutoPilot Skills

Feary, M., Sherry, L., Palmer, E., and Polson, P. (2000). Evaluation of a Formal Methodology for Developing Vertical Guidance Training Material. Presented at the Human Computer Interaction in Aeronautics Conference, Toulouse, France,(September 27-29).

Feary, M. and Sherry, L. (1998) Evaluation of a Formal Methodology for Developing Aircraft Vertical Flight Guidance Training Material. Paper presented at the 42nd Annual Meeting of Human Factors and Ergonomics Society, Chicago, IL (October 5-9).

Feary, M., Palmer, E., Sherry, L., Polson, P., Alkin, M., and McCrobie, D.(1997) Behavior-Based vs. System-Based Training and Displays for Automated Vertical Guidance. Paper presented at the Ninth International Symposium on Aviation Psychology, Columbus, OH (April 27-May 1).

Michael.S.Feary@nasa.gov

	Feary, M. and Sherry, L. Dynamic Storyboards: Faster Cheaper, Software Prototypes for Usability Evaluation. Conference on Human-Computer Interaction in Aeronautics, Toulouse, France, (29 September - 1 October).
	Feary, M., Sherry, L., Polson, P., and Palmer, E.(2001). A Formal Method for Integrated System Design. 8th European Conference on Cognitive Science Approaches to Process Control (CSAPC ’01), Munich, Germany (24-26 September).
	Sherry, L., Polson, P. and Feary, M. (2002) Designing User-Interfaces for the Cockpit: Five Common Design Errors, and How to Avoid Them. 2002 SAE World Aviation Congress, Phoenix, AZ (November 5 – 7).
	Sherry, L., Polson, P., Feary, M. and Palmer, E. (2002) When Does the MCDU Interface Work Well? Paper presented at the the International Conference on Human-Computer Interaction in Aeronautics, Cambridge, MA (October 22-24).
	Sherry, L., Feary, M., Polson, P., and Palmer, E. (2000). Formal Method for Identifying 2 Types of Automation Surprises. Honeywell Publication C69-5370-016, Phoenix, AZ, USA.
	Sherry, L., Feary, M., Polson, P., Mumaw, R., and Palmer, E. (2001). Cognitive Engineering Analysis of the Vertical Navigation (VNAV) Function. NASA/TM-2001-210915. Ames Research Center, Moffett Field, CA, USA.
	Feary, M., McCrobie, D., Alkin, M., Sherry, L., Polson, P., Palmer, E. (1998). Aiding Vertical Guidance Understanding. NASA/TM-1998-112217. Ames Research Center, Moffett Field, CA, USA.
	Sherry, L., Feary, M., Polson, P., and Palmer, E.(2001). What's It Doing Now? Taking the Covers Off Autopilot Behavior.
	Sherry, L., Feary, M., Polson, P., and Palmer, E.(2001).Why Won't It Take That? MCP Control Device Invokes More Than One Behavior.
	Feary, M., and Barshi, I. (1999) “Why is it Doing That?” Perspectives of the FMS. Paper presented at the Tenth International Symposium on Aviation Psychology, Columbus, OH (May 3 -May 7).

	Sherry, L., Feary, M., Polson, P., and Palmer, E. (2001). AutoPilot Tutor: Building and Maintaining AutoPilot Skills
	Feary, M., Sherry, L., Palmer, E., and Polson, P. (2000). Evaluation of a Formal Methodology for Developing Vertical Guidance Training Material. Presented at the Human Computer Interaction in Aeronautics Conference, Toulouse, France,(September 27-29).
	Feary, M. and Sherry, L. (1998) Evaluation of a Formal Methodology for Developing Aircraft Vertical Flight Guidance Training Material. Paper presented at the 42nd Annual Meeting of Human Factors and Ergonomics Society, Chicago, IL (October 5-9).
	Feary, M., Palmer, E., Sherry, L., Polson, P., Alkin, M., and McCrobie, D.(1997) Behavior-Based vs. System-Based Training and Displays for Automated Vertical Guidance. Paper presented at the Ninth International Symposium on Aviation Psychology, Columbus, OH (April 27-May 1).