EAC – 0018

THREAT AND ERROR MANAGEMENT

Introduction

The Threat and Error Management (TEM) model is a conceptual framework for

understanding operational performance in complex environments. Originally created to

capture the flight crew’s task in commercial aviation, the model is generic and can be

applied to numerous work situations. The added value that TEM brings to other

performance models is that it focuses simultaneously on the operating environment and

the humans working in that environment. Because the model captures ongoing

performance in its “natural” or normal operating context, the resulting description is

realistic, dynamic, and holistic. Because the model can also quantify the specifics of the

environment and the effectiveness of performance in that environment, it is also highly

diagnostic.

There are several ways of using the TEM model, from focusing on a single event (as is

the case with accident/incident analysis) to understanding systemic patterns in a large set

of events (as with LOSA). As a training tool, TEM can help individuals clarify their

performance needs and vulnerabilities, and as part of a safety management system, TEM

can help an organization measure and improve the effectiveness of its organizational

defenses and safeguards.

The Model

This section defines and provides examples of the various components of the Threat and

Error Management (TEM) model.

Threats

A threat is defined as an event or error that occurs outside the influence of the flight crew,

(that is, it was not caused by the crew), increases the operational complexity of a flight,

and requires crew attention and management if safety margins are to be maintained.

There are threats from the environment—adverse weather, airport conditions, terrain,

traffic, and ATC—and threats emanating from within the airline—aircraft malfunctions

and MEL items, problems, interruptions, or errors from dispatch, cabin, ground,

maintenance, and the ramp. Threats may be anticipated by the crew, for example, by

briefing a thunderstorm in advance, or they may be unexpected, occurring suddenly and

without warning such as in-flight aircraft malfunctions. Some threats are easily resolved

and quickly dismissed from the crew’s workload, while other threats require greater

attention and management. A mismanaged threat is defined as a threat that is linked to or

induces flight crew error.

Errors

Crew error is defined as action or inaction that leads to a deviation from crew or

organizational intentions or expectations. Errors in the operational context tend to reduce

the margin of safety and increase the probability of adverse events.

Broadly speaking, there are handling errors (flight controls, automation), procedural

errors (checklists, briefings, callouts) and communication errors (with ATC, ground, or

pilot-to-pilot). See the error management worksheet in the sample observation form,

EAC0015 Appendix A, for a more complete list of errors.

Understanding how the error was managed is as important, if not more important, than

understanding the prevalence of different types of error. It is of interest then if and when

the error was detected and by whom, as well as the response(s) upon detecting the error,

and the outcome of the error. As with threats, some errors are quickly detected and

resolved, leading to an inconsequential outcome, while others go undetected or are

mismanaged. A mismanaged error is defined as an error that is linked to or induces

additional error or an undesired aircraft state.

Threat and Error Countermeasures

A description of a flight is not complete without noting what the crew was doing to

anticipate threats and avoid errors, as well as managing those that occurred. The

following crew behaviors are considered threat and error countermeasures:

a) Planning countermeasures—planning, preparation, briefings, contingency management—are essential for managing anticipated and unexpected threats

b) Execution countermeasures—monitor/cross-check, taxiway/runway management, workload and automation management—are essential for error detection and error response

c) Review/Modify countermeasures—evaluation of plans, inquiry—are essential for managing the changing conditions of a flight

In addition to crew behaviors, TEM countermeasures also include equipment and

procedural countermeasures. Warning systems such as GPWS and weather alerts can be

considered threat countermeasures, just as checklists and well-written procedures provide

the means for error avoidance and error detection.

In sum, the TEM model captures the dynamic activity that is a flight crew planning and

executing a flight in real time and under real conditions. The utility of the model is that it

can be applied proactively or reactively, at the individual, organizational, and/or systemic

levels.

Practical Applications of the TEM model

TEM as a training tool

TEM is the foundation of human factors training programs at several airlines; TEM training emphasizes the value of threat anticipation and

management, error avoidance, and error detection and recovery. The model allows pilots

to analyze their own performance strengths and vulnerabilities. The International Civil

Aviation Organization has adopted the TEM model in its Human Factors Training

Manual (ICAO Document 9683), produced in 2002 to help airlines design human factors

curricula.

TEM concepts can be trained effectively in the classroom in the absence of LOSA.

However, TEM training can be enhanced if an airline has also conducted a LOSA. The

LOSA results can help shape the training curriculum, and pilots can discuss the findings

during training. Pilots are always interested in and respond well to data derived from their

own operation.

It is important to clarify that TEM is not CRM and should not be considered a

replacement for it. TEM and CRM refer to overlapping but not equivalent activities.

CRM refers specifically to activities conducted by the crew to optimize performance.

These activities include threat and error countermeasures such as briefing, contingency

planning, and monitor/cross-checking, but they also include higher-order concepts such

as leadership and establishing open communication in the cockpit. Similarly, TEM

includes crew countermeasures, but it also encompasses equipment, procedural and

regulatory countermeasures.

As a training tool, TEM can help individuals clarify their performance needs and

vulnerabilities from a different perspective. Hence, threat and error management

concepts could be introduced and explored as one component of CRM training.

TEM as a reporting tool for incidents

TEM has been integrated into the Aviation Safety Action Program (ASAP) Reporting forms

structured to the TEM framework instruct the pilots to describe the event at the level of

threats and errors. The TEM format prompts pilots to report information about the threats

that were present, the errors they may have made, how well the event was managed, and

how the event may have been avoided or handled better. Preliminary work has shown

that even pilots who have not had training in the TEM model are able to complete the

reporting form, a fact that speaks to the intuitive nature of the TEM framework.

21

In the ASAP environment, TEM can “go inside the pilot’s head” in a way that LOSA as

an objective observational tool cannot do and most assuredly does not want to do. With

ASAP, pilots can report personal or historical factors that contributed to the event –

information that is not privy to an observer. With LOSA, the benefit is that observers

may detect threats and errors that the crews themselves do not detect. This is one example

of how LOSA and ASAP data can complement each other at the system level.

TEM as a systematic observation tool

The TEM model was first conceived in conjunction with the development of LOSA;

hence, its original application was as an observation tool Feasibility studies are currently underway to explore the transfer of the

methodology to airline flight dispatch and air traffic control. an adapted version of LOSA called Dispatch Operations Safety

Audit (DOSA). Early results demonstrated that such a transfer of methodology is possible

and could ultimately provide a 360-degree perspective on the interaction between pilots

and dispatchers. In addition, ICAO has instituted a formal group of ATC subject matter

experts from across the world to develop the Normal Operations Safety Survey (NOSS),

a formal protocol to observe normal operations in ATC, based on the TEM model and

LOSA methodology.

TEM as a reactive analysis tool for accidents and incidents

TEM can be used as an analysis tool to understand rare events, such as accidents and

serious incidents. The IATA Safety Committee (SAC) has adopted the TEM model as an

analysis framework for its Incident Review Meetings, based on its ease of use and utility

of the extracted data.

TEM as a proactive analysis tool

When TEM is used as the framework for safety data collection, a wealth of information

can be extracted. An airline can use the data to understand patterns at the organizational

level. The data can also be collected across the industry and analyzed for systemic trends.

An analysis based on TEM can:

a) Quantify those aspects of the working environment that can pose a problem for the efficiency or safety of the operation (threat prevalence);

b) Quantify the management of those threats as either effective or ineffective (threat management);

c) Recognize high rates of threat prevalence and mismanagement as systemic vulnerabilities;

d) Codify and quantify the errors that crews commit (error prevalence);

e) Codify and quantify the error management process from diagnosis to response and outcome (error management);

f) Recognize high rates of error prevalence and error mismanagement as systemic flaws in procedures, policies, training, aircraft design, and or inter-agency coordination; and

g) Locate strengths as well as vulnerabilities in organizational safeguards.

Conclusion

The TEM model is intuitive, practical, and versatile. More and more airlines are realizing

the utility of TEM, as exemplified in the following quote from an airline manager: