LOSA
Table
of Contents
INTRODUCTION
What
is the purpose of this Advisory Circular?
To
whom does this Advisory Circular apply?
Contents
of the Advisory Circular
WHAT
IS LOSA
WHY
DO A LOSA
WHO
IS INVOLVED IN A LOSA
Departments
Pilots’
Association or Group
LOSA
Coordinator & Steering Committee
Observers
Line
Pilots
Data
Analyst and Report Writer
WHEN
TO CONDUCT A LOSA
HOW
TO IMPLEMENT A LOSA
DATA
COLLECTION
DATA
ANALYSIS & FEEDBACK
HOW
TO USE LOSA DATA
INTRODUCTION
What
is the purpose of this Advisory Circular?
This
circular provides the rationale and procedure for conducting a Line Operations
Safety Audit (LOSA) at an airline.
To
whom does this Advisory Circular apply?
This
circular is relevant for airline personnel in flight safety, flight training,
and flight operations. It is also relevant for personnel in ground operations
and in-flight services.
Contents
of the Advisory Circular
A
LOSA—introduces the LOSA process and distinguishes it from other proactive safety
data programs such as FOQA and Aviation Safety Action Program (ASAP). The next
section— why do a LOSA—lists the various types of data collected via a LOSA and
shows how they contribute to an airline’s safety management system. Next, the
personnel involved in a LOSA and their respective roles are described. The
longest section—how to implement a LOSA—is a detailed step-by-step guide to the
LOSA process from initial planning to the final report. The Circular concludes
with The Ten Operating Characteristics that define a LOSA and ensure the
integrity of the process. Under the guidance of this Circular, an airline
wanting to implement a LOSA must abide by all ten characteristics.
WHAT IS LOSA
A
Line Operations Safety Audit (LOSA) is a formal process that requires expert
and highly trained observers to ride the jump seat during regularly scheduled
flights in order to collect safety-related data on environmental conditions,
operational complexity, and flight crew performance. Confidential data
collection and non-jeopardy assurance for
pilots
are fundamental to the process.
Using
a medical metaphor, a LOSA is similar to a patient’s annual physical
examination.
People
have comprehensive check-ups in the hope of detecting serious health issues
before they become consequential. A set of diagnostic measures, such as blood
pressure,
cholesterol,
and liver function, flag potential health concerns which in turn suggest needed
changes to the patient’s current lifestyle. A LOSA is built upon the same
proactive notion. It provides a diagnostic snapshot of strengths and weaknesses
that an airline can use to bolster the “health” of its safety margins and
prevent degradation.
LOSA
is distinct from but complementary to other proactive safety programs such as
electronic data acquisition systems (i.e., FOQA), and voluntary reporting
systems (i.e., ASAP). There are two major conceptual differences. First, FOQA
and Aviation Safety action Program (ASAP) rely on outcomes to generate data.
For FOQA, it is flight parameter exceedances, and for ASAP, 4
it
is adverse events that crews report. By contrast, LOSA samples all activities
in normal operations. In these regularly scheduled flights, there may be some
reportable events, but there will also be some near-events, and importantly, a
majority of well-managed, successful flights. LOSA provides a unique
opportunity to study the flight management process, both successful and
unsuccessful, by noting the problems crews encounter on the
line
and how they manage them.
The
second major difference is the perspective taken by each program. With its
focus on electronic data acquisition downloaded directly from the aircraft,
FOQA can be said to have the “airplane perspective”. ASAP provides the “pilot
perspective” by utilizing pilots’ voluntary disclosure and self-reporting of
events. ASAP reports provide insight into why events occur, as seen from the
crew’s perspective. By contrast, LOSA provides a “neutral, third party
perspective” in that LOSA observers record contextual and flight crew data on
every phase of flight, regardless of the outcome. All three perspectives
provide useful data to an airline’s safety management system.
A
third, more pragmatic, difference between the programs relates to logistics.
FOQA and ASAP are continuous programs, i.e., they are set up to collect data on
a daily basis. A LOSA is more project-based. The full LOSA process, from
advance planning and observer selection and training, to data collection,
analyses and final report, can take six months. Despite these differences, data
from one program can be cross-referenced and used to guide data collection in
another. For example, ASAP reports may highlight a problem with departures at a
particular airport. This information can be fed to the LOSA steering committee
who can then target more observations out of that airport in order to
understand the magnitude and specifics of the problem. As another example, a
LOSA may identify a high incidence of unstable approaches, leading to a review
of procedures and the specification of new approach parameters. FOQA data can
track adherence to the
new
specifications in the interim period leading up to the next LOSA.
WHY DO A LOSA
A
LOSA provides unique data about an airline’s defenses and vulnerabilities. As
explained above, a LOSA does not replace other safety-data sources such as FOQA
or ASAP. Instead, it complements these programs and extends the reach of an
airline’s safety management system. The data collected during a LOSA can impact
almost every department in an airline, as the following examples show. The data
collected during a LOSA can help an airline:
Identify threats in the airline’s
operating environment
Observers
note events in the operational environment, e.g., adverse weather, airport
conditions, ATC clearances, terrain, and traffic congestion, and how they are
managed by flight crew. High-prevalence threats and/or threats with higher
mismanagement rates can be prioritized for further investigation; lower
mismanagement rates signify areas of strength.
For
example, understanding the extent to which certain airports or ATC practices
pose a problem for flight crews, and capturing the strategies flight crews
adopt to deal with them, can lead an airline to develop special procedures or
advisories to help its pilots manage the known threat.
1.
Identify threats from within the
airline’s operations
Observers note events arising from within the airline’s own
operations and how they are managed, e.g., operational time pressure, dispatch
errors, aircraft malfunction/MEL items, and problems with ground, ramp,
maintenance, and cabin personnel. A high number of threats arising from
dispatch or cabin might signal that these departments require attention, or
that inter-group co-operation with pilots needs to be improved, or that
procedures are inconsistent across departments. As above, prevalence and
management rates provide cues for prioritizing action.
2.
Assess the degree of transference
of training to the line
Data can be provided insight on whether training concepts
are learned, but not whether hey
are actually practiced on line.
A LOSA provides that operational information, which can be
reviewed from a training perspective to understand which areas of training, if
any, are not transferring successfully to the line.
3.
Check the quality and usability of
procedures
A LOSA provides insights about potential problems with
procedures. For example, if 5% of observed crews make a callout error during
descent/approach/land, there may be a problem with those crews. However, if 50%
of observed crews make the same error, then the evidence suggests a problem
with the callout procedure. Procedures can be ill-timed, over-long, confusing,
and/or compete for the pilots’ attention with other more important activities.
A LOSA will locate problematic procedures and policies via poor adherence
rates. A LOSA can also identify the extent of procedural deviations across
fleets.
4.
Identify design problems in the
human-machine interface
A LOSA captures aircraft handling and automation errors on
different fleets that can highlight systemic flaws in design, interface, or
adaptation. The rate at which certain errors go undetected and become
consequential can also indicate potential design vulnerabilities. An airline
can feed these LOSA findings back to the aircraft manufacturers, as well as
writing SOPs to circumvent the flaws.
5.
Understand pilots’ shortcuts and
workarounds
With experience comes expertise; pilots learn ways to save
time and be more efficient. These techniques are rarely seen in a line check,
when performance is usually done “by the book”. A LOSA provides a principled
manner by which an airline can capture collective expertise from within the
pilot group, and then share that information with all its pilots through formal
airline communication channels.
Using LOSA, false expertise—the adoption of a shortcut or
workaround that is flawed in its safety assumptions—can also be identified and
remedied.
6.
Assess safety margins
Threats and errors that are mismanaged can result in
undesired aircraft states if sufficiently serious. Vertical and lateral
deviations and unstable approaches are examples of undesired aircraft states,
also known as accident and incident precursors. A LOSA provides data about the
prevalence and management of these incident and accident precursors. Thus, an
airline acquires data about how close it is operating to the edge of the safety
envelope, without crossing the boundary into
an incident an accident.
7.
Provide a baseline for
organizational change
LOSA results provide baseline and outcome measurement data
against which organizational interventions can be measured. Using the medical
metaphor, this would be akin to the patient deciding to cut out fried foods
upon learning of a high cholesterol count. The next check-up reveals, in
quantifiable form, whether this strategy has been effective in reducing
cholesterol or whether other actions are necessary. Similarly, a follow-up LOSA
provides a new set of results which will show whether the organizational
changes were effective in reducing certain threats, errors, and/or undesired
states.
8.
Provide a rationale for allocation
of resources
Because LOSA results highlight both the strengths and
weaknesses in an organization, the results provide a data-driven rationale for
prioritizing and allocating scarce organizational resources toward
interventions.
WHO IS INVOLVED IN A LOSA
When
first exploring whether or not to conduct a LOSA, it is advisable to gather
representatives from all departments that may be potentially involved,
including flight operations, training, flight standards, the safety department,
and the pilot group.
Departments
The
flight operations and training departments typically know first-hand what is
and is not working well. These departments often have specific areas that they
would like the LOSA to focus on. Possibly the most important reason for their
involvement is that ultimately many of the problem areas that are identified
during a LOSA must be addressed by these departments. They will also be the
recipients of the potential benefits derived from the LOSA. If these
departments do not support LOSA, then there will be resistance to the findings;
however, if these departments are part of the process, there will be a sense of
ownership, and they will be invested in the results.
Pilots’
Association or Group
The
importance of having the pilots involved with and supporting the LOSA can not
be overstated. If the line pilots are convinced that their association or pilot
group supports LOSA, they will be more willing to accept the presence of observers
on their flight decks. Additionally, if pilots believe this process is
beneficial to them and to safety, they will be forthcoming and candid with
their views and safety concerns. On the other hand, if the pilots view LOSA as
a management tool to “spy on their cockpits” and they respond with “angel”
performance rather than typical performance, then the results will not be
fruitful.
Hence,
where airlines have a formal pilots’ association, leaders of the association
must be involved in the LOSA process from the beginning. If no formal pilots’
association exists, pilot representatives must be included. The pilots’
association or group can also help disseminate the results of the LOSA and
inform the pilots as to the company’s plans as a result of the LOSA.
LOSA
Coordinator & Steering Committee
Because
buy-in and support is crucial, consideration should be given to forming a LOSA
steering committee drawn from the various departments and the pilots’
association .The LOSA steering committee and/or the LOSA coordinator have many
tasks and logistical responsibilities, including:
• Publicize
the upcoming LOSA in pilot newsletters to build awareness;
• Distribute
a letter to all line pilots explaining the purpose of the LOSA;
• Decide
the size and focus of the LOSA
• Select
the observers, organize their schedules, and set up observer
training;
• Organize
a secure site for collection of the observation forms and
subsequent
data analysis.
All
of these tasks are discussed in more detail in the “How to” section of this
document.
Observers
LOSA
observers must be carefully selected to ensure the integrity of the LOSA
process. LOSA observers must be familiar with the airline’s procedures and
operations; they must also be respected by the line pilots. The observer team
can include a small number of non-pilots as long as they can anticipate and
understand flight crew tasks and their surrounding operational context;
however, the majority of the team should be active or recently retired pilots
from the airline.
In
airlines that operate more than one fleet, observers should be scheduled to
observe across fleets other than their own. This adds value to the process in
that the observer looks more at the “big picture” rather than the fine detail.
For similar reasons, experience has shown that using a small percentage of
external observers drawn from pilots with LOSA experience at other airlines
adds value in the form of a “control group” for observations. External
observers—typically 10–20% of the total observer team—will normally attend the
airline LOSA training and will need to familiarize themselves with airline
procedures. They are particularly useful in picking up systemic and
organizational threats to which airline staff are often “blind” due to
familiarity.
A
LOSA observer must be like a “fly on the wall”, able to occupy the cockpit
jump-seat and capture data without being obtrusive or interfering with the
crew’s performance. This involves creating an environment where the crew almost
does not realize they are being observed. LOSA observers will observe errors
and undesired aircraft states as part of their observations; however, they
should only interrupt and take an active role if they receive the safety of the
flight to be seriously and immediately endangered. A helpful rule of thumb is
to ask observers to think of themselves as a guest riding in the jump seat of
another airline. This seems to help distance the observer from the flight crew
while still
being
able to politely point out safety concerns if they come about.
LOSA
observers should be scheduled in the status of “passenger”, “supernumerary” or
“observer” only, i.e. they should not be scheduled as a member of the legal
operating crew. This assists both the crew and the observer to understand that
LOSA observers are data collectors only, not evaluators there to critique
crews. Observers do not have any responsibility for the operation.
Line
Pilots
A
LOSA cannot succeed without the full and candid co-operation of the line
pilots, and there can be no co-operation without trust. Line pilots must be
informed in advance about the purpose and planned implementation of a LOSA.
They
should receive a letter co-signed by credible representatives of both
management and the pilot organization that assures them of the confidential and
non-jeopardy status of LOSA data. The letter should also include a disclaimer
giving all pilots the choice of declining a jumpseat observer at their
discretion.
Only
by building in these guarantees and safeguards will the line pilots feel
sufficiently comfortable to act normally in the cockpit in the presence of a
LOSA observer. A final assurance should be an in-house publication of a summary
of LOSA results along with an outline of initial actions and proposed changes.
Data
Analyst and Report Writer
The
data analyst should have knowledge of the airline’s flight operations as well
as database management and data analysis skills. However, an airline might
choose a third party analyst if expertise is not available in-house, or if line
pilots have expressed reservations about the integrity of the LOSA
implementation or objectivity of the final report.
The
data analyst and report writer work together to prepare a report of the
findings to be presented to management and pilots.
WHEN TO CONDUCT A LOSA
There
are several factors to consider when scheduling a LOSA. Given all the personnel
involved, a LOSA should be scheduled to fit with other operational priorities.
For example, is there a particular time in the year when more observers will be
available? Is there a better time for the Scheduling department to roster these
people? Also, is there a particular time that is more interesting from a safety
or operational perspective? Some examples: bad-weather season, peak traffic
season, after the introduction of an operational change such as new aircraft,
altered routes, or a merger.
A
LOSA must not be implemented immediately after a major incident or accident.
The airline will be in a heightened state of awareness at this time, and pilots
will be overly sensitive to observation; hence, the chances of getting normal
data will be diminished. At a minimum, airlines should wait at least sex months
after a major safety event before scheduling a LOSA.
HOW TO IMPLEMENT A LOSA
This
section presents a step by step guide to implementing a LOSA. Broadly speaking,
there are steps associated with getting good-quality data from observers (Data
Collection), and steps associated with ensuring that accurate and meaningful
data are given to management and line pilots (Data Analysis & Feedback). An
airline can conduct
its
own LOSA by observing the following steps.
DATA
COLLECTION
1.
Form a LOSA steering committee and appoint a LOSA coordinator
2.
Gather information and LOSA resources from other airlines and industry groups
Before
conducting a LOSA for the first time, the committee and coordinator should seek
out information from other airlines that have already conducted a LOSA. Other
airlines may be able to share observer selection and training techniques,
observation forms, scheduling tips, and other logistical aids.
3.
Publicize LOSA within the airline and send a letter to the line pilots
A
first task is advance publicity via company publications to build line pilot
awareness and acceptance of the upcoming LOSA. Next, the coordinator organizes
and distributes a letter to all pilots explaining the purpose of the LOSA.
This
letter specifies the purpose of the audit, the fact that all observations are
of a non-jeopardy nature, and that all data will be kept strictly confidential.
The letter is signed by the highest level of management within flight
operations, with the endorsement of other relevant personnel such as chief
pilots and pilots’ representatives.
The
letter of announcement should precede the line audit by at least one month,
with a follow-up alert one week before starting observations. LOSA observers
should have copies of the signed letter to show crewmembers in case questions
arise.
4.
Decide the focus of the LOSA
The
LOSA steering committee decides the focus of LOSA. One option is to sample
broadly across the entire operation—this would be an effective strategy for a
first LOSA. Alternately, the LOSA steering committee can focus on problems that
have been identified by other data sources, such as FOQA and ASAP. This
approach would schedule LOSA observations on particular routes, in certain
regions, or into particular airports that have been identified as problematic.
The
committee can also focus a LOSA on a new fleet or other recent organizational
changes.
5.
Decide the number of observations
Most
airlines will find it cost effective to conduct a LOSA on a sample of their
operation—the question is how big a sample?
As
a general guideline for a full LOSA, match the number of observations per fleet
to the relative number of departures per day. For example, if 30% of departures
occur on Fleet A, then approximately 30% of the LOSA observations
should
occur on Fleet A. Within each fleet, try to sample as many different crews
as
possible, and as a rule, conduct 50 or more observations per fleet. Below that
number,
there is the risk of not accurately capturing a representative sample.
Modify
the guideline slightly when focusing on a particular operation or region.
For
example, to sample international flights into a particular sub-continent, then
regardless
of what percentage they constitute of the airline’s daily departures, still
schedule
at least 50 observations to ensure a good sample.
6.
Create an observation form
The
observation form should be based on a conceptual framework that captures
multiple
aspects of normal operations, including the operating environment and
flight
crew performance. It should provide categories and codes to streamline
observations
and save the observer’s time, but it should also require a written
description
of the flight that captures the full context. See appendix (A) sample LOSA
observation form
7.
Select observers
The
observer team should have representatives from flight operations, training,
safety,
and the pilots’ association. Some airlines employ a selection procedure
whereby
management and the pilots’ association each put forth a list of acceptable
observers,
and then those who appear on both lists are selected.
The
number of observers needed depends on the size of the audit and the
observers’
workload. There is substantial work involved in completing an
observation
form and providing a detail-rich narrative for each flight; therefore,
the
recommended number of observations is 10-15 per observer, depending on
routes
and schedules. Hence, a LOSA that plans 150 domestic observations
requires
at least 10 observers, while a 300-observation LOSA that includes
international
flights requires 20-25 observers.
8.
Train observers
LOSA
observers must be educated about the purpose and rationale of LOSA, and
trained
in the use of the observation form. LOSA observer training typically takes
two
to three days. To assist in the design of the training, members of the steering
committee
may want to attend LOSA observer training at another airline first,
Observers
should practice with scripted scenarios or videos until they are
confident
they can use the observation form correctly. At this point, they can be
dispatched
to the line; however, it is recommended that observers be brought back
in
after one or two flights, to discuss their observations, correct any
misperceptions,
and coach them on areas that require clarification. Appendix C
provides
more detail on the objectives and content of observer training.
9.
Schedule observations
Plan
no more than two observations per observer per day to allow sufficient time
to
complete the observation form and write a rich narrative. Schedule observers
across
fleets regardless of their type rating to encourage a more general, crossfleet
perspective
of flight crew performance. Build some flexibility into the
schedule
to allow for the unexpected. Finally, do not let the observations continue
indefinitely—schedule
all observations within a 1-3 month period if possible, else
the
impact of LOSA will be lost.
10.
Decide on a data repository
The
LOSA coordinator organizes a secure site for the data and oversees the
receipt
of the observation forms. The coordinator must be able to protect the
identity
of the observers and the observed to ensure complete confidentiality and
non-jeopardy
conditions. Under no circumstances should it be possible to connect
individuals
with particular observations.
The
observations can be kept in-house if data management and analysis expertise
is
available, and if data security can be assured. Alternately, the data can be
sent
to
a trusted third party who will assume responsibility for data collection,
cleaning,
and analysis. The decision will depend on airline resources and pilot
trust
issues.
1
There have been several LOSA conferences in different
regions of the world. At these events, airline
representatives learn about Threat and Error Management,
they hear from other airlines who have
implemented a LOSA, and they attend an overview of LOSA
observer training using the TEM model.
11.
Provide logistical support
Give
the observers the name of a contact person, most likely the LOSA
coordinator,
who can be reached if there are any problems with scheduling or data
collection.
DATA
ANALYSIS & FEEDBACK
12.
Verify the data
Convene
a meeting of “local experts”—airline personnel familiar with the
operation
of each fleet (possibly fleet managers or member of the steering
committee,
but not any of the observers). The group’s task is to review and verify
the
observations against current manuals, policies, and procedures. For example,
an
observer might log a procedural error for failure to make an approach callout
when
in fact there is no written procedure in the airline’s flight operations
manual.
The
data verification group would delete this particular ‘error’ from the database.
This
step is a data integrity check in that it ensures that events are correctly
recorded
in line with each fleet’s procedures and policies. It also builds ownership
in
the results and dispels any later criticism that the coding was not an accurate
representation
of the airline’s operations.
13.
Analyze data
LOSA
data reveal strengths and vulnerabilities in an airline’s operations. The data
analyst
should investigate the prevalence and management of different events and
errors.
Although certain types of comparisons will seem obvious, many analyses
can
and should be based upon hunches and theories derived from local knowledge
of
operations. If the analyst knows how fleets and operations are managed,
comparisons
that reflect this structure can be made. If the analyst knows the kinds
of
information that might be useful to training, safety, or to domestic or
international
flight operations, results can be tailored to these particular aspects of
the
operation. Feedback from various airline stakeholders is critical during this
stage
of preparing the report. The analyst should not hesitate to distribute early
drafts
to key people within the airline familiar with LOSA to cross-verify the
results.
This not only helps validate derived trends, but it gives other airline
personnel
ownership of the report.
Patterns
will emerge as the data are analyzed. Certain errors occur more
frequently
than others, certain airports or events emerge as more problematic than
others,
certain SOPs are routinely ignored or modified, and certain maneuvers
pose
greater difficulty for adherence than others. These events and practices form
the
basis of suggested targets for enhancement.
14.
Prepare report
The
last stage of a LOSA is a written report that presents the overall findings of
the
audit. With a large database like the one generated from a LOSA, it is easy to
fall
into the trap of trying to present everything. The report must be concise and
present
only the most significant trends from the data.
Along
with the results, the report should provide an initial list of targets for
enhancement.
Targets need to be action-focused and data-driven. Some example
targets
that might emerge from a LOSA include:
a) Reduce
the number of unstabilized approaches
b) Streamline
pre-departure checklists
c) Reduce
SOP cross-verification errors
d) Understand
automation errors on the new fleet
e) Investigate
conditions at airports X and Y
f)
Improve management of adverse weather threats
g) Investigate
high rate of MEL items on the ZZ fleet
h) Reduce
dispatch errors at the hub
i)
Develop an international flight operations guide
j)
Develop a module on intentional non-compliance errors for
Captain
k) Upgrade
training
15.
Brief management
The
LOSA report should be presented to management in operations, training,
standards,
safety, and possibly other departments depending on the results. For
example,
representatives from ramp, maintenance, dispatch and cabin may want
to
hear how their work is perceived from the pilots’ perspective, particularly if
it
is
problematic. A briefing to the pilots’ association is also recommended.
Once
the various departments are briefed on the report, they will likely want to
investigate
the data more deeply themselves. The data should be available in
aggregated
form for them to review. Some flight narratives will also be of
interest,
hence the prerequisite insistence on de-identifying the observations.
16.
Brief line pilots
Line
pilots should also be informed of the significant results in the LOSA report.
To
sustain the pilots’ interest in the LOSA project, make an announcement at the
end
of the data collection phase that the LOSA observations have been completed,
stating
how many and on what fleets, and advise when the pilots can expect to see
the
results.
When
the report is ready, the highlights should be presented to the pilots, either
as
one
LOSA debriefing event or spread over time in the airline newsletter or other
safety
periodical. Pilots will want to know what changes will be undertaken as a
result
of the LOSA.
17.
Monitor safety change process
Historically,
organizational safety changes within airlines have been driven by
accident/incident
investigation and intuition. Today, airlines must deal proactively
with
accident and incident precursors. To be successful, the safety change process
must
be data-driven. Measurement of daily operations is fundamental, because
unless
an organization uses systematic measurement, the perspective it has on the
strengths
and weaknesses of its operations is largely based on anecdote and
opinion.
A
LOSA provides specific and quantified results. To take full advantage of this
specificity,
the targets for enhancement that arise from the data analysis should go
through
a formal safety change process to produce improvement. A formal safety
change
process provides a principled approach to target limited resources and
helps
the airline avoid “turf” issues, by clearly defining and prioritizing the
issues
that
impact flight operations. The basic steps of a safety change process are:
a) Measurement
(with LOSA) to obtain the targets;
b) Detailed
analysis of targeted issues;
c) List
of potential changes for improvement;
d) Risk
analysis and prioritization of changes;
e) Selection
and funding of changes;
f)
Implementation of changes;
g) Time
for changes to stabilize;
h) Re-measurement.
HOW TO USE LOSA DATA
A
well-conducted and well-analyzed LOSA identifies strengths and vulnerabilities
in an
airline’s
operations. It provides this information in a quantifiable form against which
targets
can be specified and improvements can be measured. The following example
briefly
illustrates the step-by-step integration of LOSA data into the safety change
process.
An
airline’s LOSA results indicate that 16% of observed flights involved an
unstable
approach. Because observations were scheduled across the operation,
and
the number of observations exceeded 50 per fleet, the LOSA committee is
confident
that the percentage is an accurate representation of operations as a
whole.
Following
management briefings and extensive discussion, a specific target for
improvement
is created to “reduce the number of unstabilized approaches by
50%,
that is, reduce the number of unstable approaches from 16% to 8% or fewer
of
all landings.”
An
action committee is formed for unstabilized approaches. They formalize the
parameters
and definition of an unstable approach, they review existing
procedures
and training, and they introduce changes in all relevant areas.
A
repeat LOSA is conducted three years after the first LOSA. The data, once
aggregated
and analyzed, show the new rate of unstable approaches to be 12%.
The
airline concludes that changes made to the operation were successful in
reducing
the rate of unstabilized approaches from 16% to 12%, an improvement
of
25%. Upon reviewing the results of the second LOSA, the airline recommits to
its
original target of reducing the unstable approach rate to 8% or lower, and
continues
to focus efforts in this area.
Depending
on the sophistication of an airline’s safety management system, and the extent
to
which different safety programs within the airline are premised on the same
conceptual
framework,
data from a LOSA can be cross-referenced with data from the ASAP or
FOQA
programs. Each data source provides unique yet complementary evidence of the
airline’s
safety status. In the above example, the airline might track unstable approaches
through
its FOQA program using new flight parameters decided by the action committee
and
then implemented into procedures and training. To see if pilots are incurring
problems
with the new procedure, the FOQA aircraft data can be cross-referenced with
ASAP
reports of events resulting from unstable approaches. This way, the airline
does
not
have to wait until the next LOSA to learn if its interventions are being
successful.
LOSA
data are useful in another way. LOSA presents a broad view of operations; a
repeat
LOSA can maintain that broad focus. For example, did the changes that were
introduced
after the first LOSA improve results in one area, only to cause problems in
another?
Checklist adherence may have improved, but did error detection—the
superordinate
goal of improving checklist adherence—actually improve or is the new
adherence
simply cosmetic?
SUMMARY: THE TEN OPERATING CHARACTERISTICS OF
LOSA
Ten
operating characteristics define and summarize the LOSA process. Under the
guidance
of this Advisory Circular, a LOSA must observe all ten characteristics to
ensure
the
integrity of the LOSA process and the quality of the final product. These ten
characteristics
have been formally endorsed by ICAO, IATA, and US ALPA.
The
Ten Operating Characteristics are:
1.
Jump-seat observations during normal flight operations
LOSA
observations are limited to regularly scheduled flights. Line checks, initial
line
indoctrination or other training flights are off-limits due to the extra level
of
stress
put on pilots during this type of situation. Having another observer on board
adds
to an already high stress level, thus providing an unrealistic picture of
performance.
In order for the data to be representative of normal operations,
LOSA
observations must be collected on regular and routine flights.
2.
Joint management/pilots’ association sponsorship
In
order for LOSA to succeed as a viable safety project, there needs to be support
not
only from the management side but also from the pilots. The joint sponsorship
provides
a “check and balance” for the project to ensure that change, as necessary,
will
be made as a result of LOSA data. When considering whether to conduct a
LOSA,
the first question to be asked by airline management is whether the pilots’
association
(or pilot group representatives) endorses the project. If the answer is
“No”,
the project must not be initiated until endorsement is obtained.
3.
Voluntary crew participation
Maintaining
the integrity of LOSA within an airline and the industry as whole is
extremely
important for long-term success. To accomplish this goal, all LOSA
observations
are collected with voluntary crew participation. Before conducting
LOSA
observations, observers must first ask the flight crew for permission to be
observed.
If the crew declines, the observer takes another flight with no questions
asked.
If an airline conducting a LOSA has an unreasonably high number of
declines,
this should serve as an indicator that there are critical “trust” issues to be
resolved.
4.
De-identified, confidential, and non-disciplinary data
collection
LOSA
observers are required not to record names, flight numbers, dates, or any
other
information that can identify a crew or individual. The purpose of LOSA is
to
collect safety data, not to punish pilots. Airlines cannot allow themselves to
squander
a unique opportunity to gain insight into their operations by having
pilots
fearful that a LOSA observation could be used against them for disciplinary
reasons.
If a LOSA observation is ever used for disciplinary reasons, the
credibility
of the entire safety program may be irreparably compromised.
5.
Targeted observation form
The
LOSA observation form is predicated on the TEM framework. At the
airline’s
own initiative (and risk), other conceptual frameworks can be used for
LOSA
data collection. Whatever framework is used, it must generate meaningful
data
on a variety of topics, including what the crews did well, what they did
poorly,
and how they managed each phase of flight. A narrative written by the
observer
should have sufficient detail to allow others to understand the flight and
all
its events. The observers need to describe the environmental conditions and
events
surrounding the pilots’ behavior so that the crews’ performance can be
understood
in full context.
6.
Trained and calibrated observers
Primarily,
pilots conduct LOSA. Observation teams will typically include line
pilots,
instructor pilots, safety pilots, management pilots, and representatives of
the
pilots’ safety committee. It is critical to select observers that are respected
and
trusted
within the airline to ensure line acceptance of LOSA. After observers are
selected,
they are trained and calibrated in the LOSA methodology, including the
use
of the LOSA observation form. Observers’ training in the concepts and
methodology
of LOSA will ensure that the observations will be conducted in the
most
standardized manner. (See Appendix C.)
7.
Trusted data repository
In
order to maintain confidentiality, airlines must have a trusted data
repository.
This
site can be in-house, such as that used for other confidential data such as
FOQA,
or it can be off-site. The goal is that no individual observations will be
misplaced
or improperly disseminated through the airline.
8.
Data verification
Data-driven
programs like LOSA require quality data management procedures
and
consistency checks. For LOSA, these checks are done at data-verification
roundtables.
A roundtable consists of three or four department and pilots’
association
representatives who review all the raw data for possible inaccuracies.
The
end product is a database that is validated for consistency and accuracy
according
to the airline’s standards and manuals, before any statistical analysis is
performed.
9.
Targets for enhancement
The
final product of a LOSA is the data-derived targets for enhancement based on
emergent
patterns in the data. It is then up to the airline to develop an action plan
based
on these targets, using experts from within the airline to analyze the targets
and
implement appropriate change strategies.
10.
Feedback results to the line pilots
In
order to ensure long-term success of LOSA, airlines must communicate the
results
back to the line pilots. Pilots will want to see not only the results of the
audit,
but also management’s plan for improvement.