1. AYR COLLEGE
Flight Deck Evolution – The
Path to Automation and
Beyond
HND Aeronautical Engineering Graded Unit
Jon Galbraith
S10172585
2011/2012
An appraisal of flight deck instrumentation, from mechanical and electro-mechanical instruments,
through to modern TFT displays and beyond. Modern voice recognition technology will be discussed and
assessed for suitability in the modern flight deck.
2. ~ 1 ~
Table of Contents
FLIGHT DECK EVOLUTION – THE PATH TO AUTOMATION AND BEYOND .......... 2
PROJECT BRIEF..........................................................................................................................................................2
PROJECT SPECIFICATION AND GANTT CHART............................................................................................................3
AN INTRODUCTION ....................................................................................................................................................5
INSTRUMENTS – THE BASICS .....................................................................................................................................6
ELECTRONIC DISPLAYS ...........................................................................................................................................11
THE INTEGRATION OF A SYSTEM .................................................................................... 15
REQUIREMENTS .......................................................................................................................................................15
SAFETY ASSESSMENT ..............................................................................................................................................16
WHAT IS CERTIFICATION? .......................................................................................................................................19
TESTING PRINCIPLES ...............................................................................................................................................22
INTEGRATION OF VOICE RECOGNITION SOFTWARE............................................... 24
THE REQUIREMENT .................................................................................................................................................24
SAFETY ASSESSMENT ..............................................................................................................................................24
SYSTEM DESIGN ......................................................................................................................................................29
HUMAN FACTORS – STRESS.....................................................................................................................................31
TEST PROCEDURE - METHODOLOGY........................................................................................................................32
TEST PROCEDURE - SUMMARY OF FINDINGS ...........................................................................................................37
FLIGHT DECK EVOLUTION – THE PATH TO AUTOMATION AND BEYOND ........ 39
A CONCLUSION........................................................................................................................................................39
REFERENCES............................................................................................................................................................40
APPENDICES
3. ~ 2 ~
Flight Deck Evolution – The Path to
Automation and Beyond
Project Brief
This project will look at the evolution of flight deck instrumentation and the possible
integration of new technology. The project will include:
- Appraisal of flight deck instrumentation
- Mechanical and Electro-mechanical instruments
- ‘Glass Cockpit’ and TFT Displays
- Developments and limitations of flight deck information
- Voice recognition software advantages/ disadvantages
- Human factors including stress
- Testing of software and results
As a subset of the overall remit of the project, modern technology, in particular voice recognition
and voice response software will be discussed and assessed for suitability in the modern flight
deck. The development of this software will be discussed with particular regard to safety,
certification and any issues faced with possible integration. A practical exercise in testing the
software will further assess the suitability of the software before a conclusion is made.
It is expected that the project shall take approximately thirty-one weeks to complete.
4. ~ 3 ~
Project Specification and Gantt Chart
The majority of this project will be compiled of research into flight deck instrumentation
from the past and present, voice recognition software, testing and certification of new
technology. The project specification is as follows:
Section A: Flight Deck Evolution Introduction and Current Instrumentation
Section B: Integration of a System
Section C: Integration of a Voice Recognition System (including test procedure)
Section D: Conclusion and References
6. ~ 5 ~
An Introduction
The aviation industry has fast become one of the most technologically advanced
industries in the world with ever increasing automation at the very forefront of the industry’s
continuing evolution. Aircraft have come a long way since the Wright brothers first took to the
skies back in 1903. Modern day rivalries between the likes of Boeing and Airbus continue to
contribute positively to the emergence of new technologies, and on-going space and military
research constantly supply new ideas and innovations to the industry. Through the years, aircraft
have become more aerodynamic, grown in size, flown faster, reached new heights, but perhaps
most importantly, become safer. In addition to producing more structurally sound,
aerodynamically, economically and environmentally efficient aircraft, the manufacturers have
also continued to modernise and develop one of the more major and important aspects with
regard to safety – the Flight Deck. Through the years there has been the transformation of
instruments from mechanical to electro-mechanical, and more recently from electro-mechanical
to ‘glass cockpit’. Automation is ever increasing as is the production of new technologies. With
the recent emergence of Voice Recognition aboard the Typhoon military jet, it would be thought
that such developments would also be made within the commercial aviation industry. However, a
lot of work has to be carried out when considering the integration of a system, it is not quite as
easy as ‘the system works, let’s pop it in’, there are lots of procedures to be completed before
integrating a system. These include requirements writing, safety assessment analysis, human
factors considerations, certification and rigorous testing of the system. Therefore, once these
procedures are completed, would the integration of a Voice Recognition system aboard
commercial aircraft actually be practical?
7. ~ 6 ~
Flight Deck Instrumentation
Instruments – The Basics
A standard instrument always consists of three main components:
- A sensor
- A processor
- An indicator
The senor is sensitive towards the measured parameter. It converts the parameter measured
into a quantity which is simple and easy to process. The processor takes the parameter input from
the sensor and prepares it for indication. The processor accounts for factors such as calibration,
fault detection, fault correction and signal amplification. Finally, the indicator displays the
measured parameter in the correct format. These three components are traditionally situated
within the same housing. Over time, however, both the processor and sensor were located within
an analogue or digital computer, away from the indicator housing.
Figure 1: Simple Flowchart of Instrument Components
Modern day developments include the positioning of the sensor at a location where the
conditions for measuring the parameter are ideal. A data bus has also been incorporated to send
the signals to a digital processor, this forms part of a computer network. The signals are then
processed and sent to the dedicated aircraft instruments, systems and displays. The instruments
themselves also feature glare shield technology. A glare-shield is a colourless shield which
covers the main instrument panels and protects them from direct sunlight.
Sensor Processor Indicator
8. ~ 7 ~
The vast majority of modern day passenger aircraft have a similar layout of their instruments
in specific instrument panels. Radio and communication controls are found between the pilot and
co-pilot seats on a pedestal. The overhead panel houses the control panels for all of the aircraft’s
systems. Facing the flight crew is the main instrument panel. The uppermost central part mostly
houses the autopilot controls. The centre main instrument panel houses engine indicators. The
left and right main instrument panels house the instruments like airspeed indicator, altimeter etc.
and is identical for both pilots. A basic image of this can be found below in Figure 2.
`
Figure 2: Aircraft Instrument Panels
For Figure 2 the acronyms are as follows:
- OHP – Overhead Panel
- MCP – Mode Control Panel
- LMIP – Left Main Instrument Panel
- RMIP – Right Main Instrument Panel
- CMIP – Centre Main Instrument Panel
- PED – Pedestal
OHP
MCP
LMIP RMIPCMIP
PED
9. ~ 8 ~
The basic ergonomic instrumentation principle is the ‘basic t-shape’ or ‘basic 6’
arrangement. It houses the six fundamental instruments:
- Artificial Horizon
- Heading Indicator
- Airspeed Indicator (ASI)
- Attitude Indicator
- Vertical Speed Indicator (VSI)
- Altitude Indicator
- Rate of Turn Indicator
Figure 3: Basic T-Shape Instrument Layout
The earliest flight deck instrumentation examples featured an endless amount of analogue
instruments all in very close proximity to each other, literally side by side. An example of the
flight deck of a Boeing 707 type aircraft can be seen overleaf in Figure 4.
10. ~ 9 ~
Figure 4: Flight Deck of a Boeing 707
It can be seen in Figure 4 that there are two ‘basic-6’ arrangements, one for each pilot. In
the centre, the analogue engine instrumentation can be seen. To anyone without any aircraft
knowledge, or even those with some basic knowledge, the earliest flight deck examples may
seem a bewildering concept and the demand for pilots to be in control and supervise the
configuration of the aircraft must have been very high.
As evolution in the aircraft industry began, with airliners being the way forward, the
flight deck instrumentation progressed too, and although still very much an ‘analogue’ flight
deck, the example seen in Figure 5 overleaf, of a Boeing 737-200, already seems simpler and de-
cluttered somewhat.
11. ~ 10 ~
Figure 5: Flight Deck of a Boeing 737-200
Figure 5 shows the most modern example of an analogue flight deck and its furthest
stage of development in the mechanical flight deck instrumentation phase before the birth of the
electronic era.
12. ~ 11 ~
Electronic Displays
As the aviation industry moved forward, the technology used in the flight deck
progressed also, with the introduction of electronic display methods. There are several different
types of electronic display:
- Incandescent (Earliest)
- Light Emitting Diode (LED)
- Cathode Ray Tube (CRT)
- Liquid Crystal Display TFT (LCD)
Below in Figure 6 is an example of a typical seven-segment incandescent display.
Figure 6: Seven-Segment Display
An incandescent display is an electronic display used for displaying numerical characters
and decimals. The most common incandescent displays feature seven segments; hence they are
named seven segment displays. For the ease of reading, they are often, as shown in Figure 6,
slanted slightly. Their earliest principle of operation concerned the illumination of incandescent
light bulbs housed behind each of the seven segments.
13. ~ 12 ~
More modern seven segment displays use LEDs (Light Emitting Diodes) for illuminating
the seven segments. In Figure 6 on the previous page, all of the 7 segments are illuminated,
producing a number 8. The decimal above is not illuminated in this case. LEDs work by
producing light when conducting electricity. The colour of an LED can be easily altered by
simply changing the cover placed over the segment. Examples of LEDs can still be found in the
flight deck today on, for example, the autopilot control panel as shown below in Figure 7.
Figure 7: LEDs on Boeing 737 Mode Control Panel (MCP)
14. ~ 13 ~
The next step was the introduction of the CRT (Cathode Ray Tube) display system. This
began the EFIS (Electronic Flight Instrument System) principle, where information is displayed
on screens, with analogue instruments used only as a back-up. The displayed parameters
displayed on the EFIS are the: PFD (Primary Flight Display) which shows the artificial horizon,
heading, airspeed, altitude and vertical speed; the MFD (Multi-Function Display) which displays
information about the weather and navigational principles; and the EICAS (Engine Indicating
and Crew Alerting System) which displays all relevant engine instrumentation such as
temperatures, fuel levels and alerting systems should any problems occur. The principle of a
CRT itself, concerns the operation of an electron ‘gun’ firing at a fluorescent screen to produce
an image. Most televisions used this technology before the invention of LCDs (Liquid Crystal
Displays) and plasma technology. A diagram of a CRT can be seen in Figure 8 below with a
description following.
Figure 8: Cathode Ray Tube
The principle of the CRT is the operation of the negatively charged cathode and heater system.
The heater when active produces electrons, and due to the cathode also being negatively charged,
the electrons are repelled away from it in the direction of the fluorescent screen. The anodes
increase the velocity of the electrons due to their opposite charge. The deflecting coils then direct
the electron flow to hit the desired part of the fluorescent screen and this proportion of the screen
lights up. The fluorescent screen is made up of Phosphor dots or lines, which are coloured red,
green and blue. These 3 basic colours form the basis of all colours that can be produced by the
screen when mixed adequately.
15. ~ 14 ~
The most modern development is that of the LCD (Liquid Crystal Display). The LCD
uses liquid crystals and the modulation of light to produce images on a screen. The liquid crystal
solution is held in between two transparent screens, through which light is projected through.
The light is either allowed to pass through the solution, or is blocked by it, thus producing the
images. They are of poorer quality from a resolution point of view, compared to CRTs but are
lighter and easier to flat-pack into a smaller area. In addition to this, LCD TFT displays are the
latest to be developed. These are essentially LCD screens that feature a TFT (Thin Film
Transistor). The thin film transistor improves the quality of the produced image, and hence are
features of the majority of EFIS’s integrated into modern aircraft. An example of an LCD TFT
can be seen in the Boeing 787 flight deck shown in Figure 9 below.
Figure 9: LCD TFT Screens on Boeing 787
It is due to this ever increasing technology that the pressure on pilots to monitor all
systems have decreased, and with aircraft becoming ever more increasingly automated, it is
important that adequate instrumentation is also modified to ensure that the aircraft remains as
safe as possible. It could be thought that increasing modifications and progression in HUDs
(Head-Up Displays) would be the next step in the progression of flight deck technology.
16. ~ 15 ~
The Integration of a System
Requirements
In reality, a system would not be integrated into the flight deck of the aircraft because it is
deemed a good idea. It must contribute positively in some way to the operation and/or
performance of the aircraft and/or personnel on board. There must be, therefore, a ‘requirement’
for the system to be there in the first place. Therefore, a set of System Requirements would be
compiled, usually by way of the customers stating ‘this is what we want and why’, or by way of
the manufacturer trying to sell the product in stating ‘this is why you need this package.’ The
requirements themselves need to be properly evaluated in terms of being; correct, feasible,
necessary, of a certain level of priority, non-ambiguous and finally verifiable (See Figure 10).
Thus, each requirement must be analysed, verified, well documented and derived.
Requirement System Requirements Definition
Correct Must contain a correct description of the intended functionality of the system
Feasible Must be realistic in terms of system integration into the intended environment
(Shows an acknowledgement of both capabilities and limitations)
Necessary Must outline the real requirement for the system in terms of why it should be
integrated
Prioritised Must outline the importance of the system in terms of essentiality
Unambiguous Must be no misunderstanding or misinterpretation of what the requirement is
Verifiable Must show the ability to perform tests and demonstrations of the system and
concludes whether these requirements are properly demonstrated in the system
Figure 10: Requirement definitions
17. ~ 16 ~
Safety Assessment
Safety Assessment is perhaps the most important consideration when integrating a
software package into an aircraft. After all, if the system is unsafe, there is a high risk of
catastrophic loss of the aircraft and its occupants. There are differing methods of safety
assessment procedures, examples being FHAs, FMEAs and FMECAs.
An FHA (Functional Hazard Assessment) is a technique used to investigate the effects of
functional failures of parts of the system. The primary intention of an FHA is to identify the
hazardous function failure scenarios. The method of conducting an FHA is relatively
straightforward:
- For a suitable process, select functions in turn
- Define the behaviour and purpose(s) of the function
- Consider and cater for hypothetical failure modes*
- Determine the effects of these
- Determine associated risk factors**
- Record associated risk factors
- Justify the associated risk factors
*Failure modes include ‘Loss of function’, ‘Operation when not required’, ‘Incorrect operation’
etc.
**Risk factors include ‘sensitivity’ and ‘probability budget’
FHA results are usually displayed in a table as shown below:
Function Failure Condition Phase Effect Class Verification
- - - - - -
Figure 11: FHA Table
18. ~ 17 ~
Another two methods which are very similar to one another are FMEAs and FMECAs.
An FMEA (Failure Mode and Effects Analysis) is a systematic approach to the identification of
risks and an analysis of determining possible failure modes whilst trying to prevent their
occurrence. An FMECA (Failure Mode, Effects and Criticality Analysis) however is basically an
extension of an FMEA, as in addition to covering the basics of an FMEA, it also takes into
account a criticality analysis. A criticality analysis is used to record the likelihood of failure
modes occurring against the severity and impact of their consequences:
‘The result highlights failure modes with relatively high probability and severity of
consequences, allowing remedial effort to be directed where it will produce the greatest value’
Figure 12: Quoted Explanation of FMECA
In addition to the above, the four fundamental aspects of a FMECA are as follows:
- Identification of Faults (i)
- Potential Effects the Fault can Cause (ii)
- Existing and/or Projected Control and/or
Compensation (iii)
- Summary of all Findings (iv)
(i)- Identifies any possible hazardous conditions that can develop
(ii)- Explains why there is a problem as a result of the condition
(iii)- Describes the actions which have to be taken in order to compensate for and/or control the
problem
(iv)- Conclusively states whether the situation is under control or whether further action is
required
The effects of a system can be defined as negligible, marginal, critical or catastrophic.
These effects can also be seen in the Criticality Level Pyramid which is shown overleaf in Figure
13 and a description follows.
19. ~ 18 ~
Figure 13: Criticality Level Pyramid
Level A - Catastrophic: often results in complete loss of the aircraft, passengers and
crew. It can happen by placing too much of a workload on the crew. An example would be a
function or system failing and all of a sudden the pilots could be too focussed on dealing with the
breakage that a catastrophic failure then occurs. The catastrophic event doesn’t necessarily result
from a faulty indication or failure of a system that removes important data, it could be that the
initial breakage could cause so much of a problem in terms of correction, that the pilots cannot
maintain control over the aircraft and the situation at that time.
Level B – Hazardous/Severe: may not involve the complete loss of life but can
potentially lead to such a serious condition.
Level C - Major: can cause loss of control of the aircraft and potentially cause injuries to
passengers and crew.
Level D - Minor: can cause some sort of detrimental effect, however the aircraft can often
overcome the situation and pilots can maintain control of the aircraft.
Level E – No Effects: results in no noticeable effect. As discussed earlier the differing
system safety assessment processes identify the criticality levels. If Level E is declared, then
there is a requirement for some sort of justification either by the manufacturer or customer.
20. ~ 19 ~
What is Certification?
Certification is a means of protection. Certification ensures that the package considered
for integration provides security and means of protection. In other terms, it should not detriment
the safety of the aircraft, its occupants, the ground upon which it operates and the ground which
it flies over. It must not cause any potential dangers to any products or goods carried within
and/or within the vicinity of the aircraft. There should be no interference caused to external
equipment or systems.
There are two main parts to certification of a system. There is the TSOA (Technical
Standard Order Authorisation) and the TC (Type Certificate) or STC (Supplemental Type
Certification). The TSOA concerns the development of an aircraft avionic appliance. This can
include the likes of Weather Radars, Radio Communication Devices and other ‘boxes’ like
EGPWS (Enhanced Ground Proximity Warning System) etc. The TSOA concludes whether a
system is permitted for use on an aircraft in that it meets the minimum standard required for
installation. After approval with regards to the TSOA however, the system is not given the go
ahead just yet. Next, the system must comply with either a TC or an STC. Both are similar in that
they concern the actual system installation onto the aircraft. They differ in that a TC is with
regards to a new aircraft, whereas an STC is for the addition of a new system onto an already
existing aircraft that is already in service.
There is a lot more factors of consideration in certification than just TSOAs and STCs
though. With regards to integrating systems involving software, for example, a PSAC (Plan for
Software Aspects of Certification) must be compiled, which is one of Five Key Plans which will
be discussed later.
A PSAC is quoted as:
‘The primary means used by the certification authority for determining whether an
applicant is proposing a software life cycle commensurate with the rigor required for the level of
software.’
Figure 14: PSAC Definition Quote
21. ~ 20 ~
In other words, a PSAC is the first stage in concluding whether the proposing body, is
putting forward a software development structure which is of an adequate severity for the
software intentions. It is generally around 20-30 pages in length and features certification
considerations, a system overview, and a software overview (See Figure 15 below). It will also
discuss system architecture, criticality and safety with some referencing to any other designs
and/or equipment. It should then be submitted to and approved by EASA (European Aviation
Safety Agency).
PSAC Sections PSAC Sections Explanation
Certification
Considerations
A summary of the certification principle which includes aspects such as
means of compliance relating to SAC (Software Aspects for
Certification). It should also define the proposed software level(s) and
includes the conclusion from the SSA (System Safety Assessment)
process, with the inclusion of possible software contribution to any
conditions involving failure
Software Overview Features a brief specification of the software functions with particular
concentration on proposed safety and partitioning concepts for example:
‘resource sharing, redundancy, multiple-version dissimilar software,
fault tolerances and timing/scheduling strategies’
System Overview Provides a system overview which should include a depiction of the
system functions and their distribution to both the hardware and software.
It should also provide information regarding safety features, hardware
interfaces, software interfaces system composition and any processing
units used
Figure 15: PSAC Sections and their Explanation
22. ~ 21 ~
The second of the Five Key Plans is QA (Quality Assurance). This addresses the role
which QA plays throughout the whole development process from beginning to end. It also
ensures that all planning is coordinated and followed. It ensures that any criterion for transition is
conformed to. It should also: ‘address conformity reviews and inspections whilst providing
guidance and timelines for audits/reviews by QA (including checklists.)’
The third of the Five Key Plans is CM (Configuration Management). It concerns the
‘tight configuration control of the product, data item documents and artefacts of the lifecycle.’ It
will also depict the numbering and naming protocol along with the approach to revision.
The fourth of the Five Key Plans is SWDP (Software Development Plan). The SWDP
concerns all aspects involved in development including personnel, scheduling, deliverables,
reliance, stages of development and any involved organisations. It will also address the
development, surrounding environment and any equipment used. Lifecycle processes should be
mentioned also i.e. ‘requirements, design, coding, integration, testing, changes etc.’ It will
address the roles of the QA and CM and any deliverables.
The fifth and final aspect of the Five Key Plans is SWVP (Software Verification Plan)
addresses acceptance and analysis of the software, traceability and virtually every aspect of
testing and actual integration of the software. It will also consider the test surroundings and
equipment, re-verifications and reversion and also any resource materials and the project
management.
1.PSAC 2.QA 3.CM 4.SWDP 5.SWVP
Figure 16: Five Key Plans
23. ~ 22 ~
Testing Principles
There are four standard types of test procedure when it comes to performing software
tests:
- Functional
- Normal Range
- Robustness
- Structural Coverage
Functional tests are those which cover every performance aspect of the software in terms
of functionality i.e. Does the software do absolutely everything that it is designed to do?
Functional tests cover everything from Low-Level tests to High-Level critical tests.
Normal Range testing is different in that it tests the functionality in terms of what the
design is. It tests that the software performs the tasks that it is intended for under normal
operating conditions. These tests can sometimes exercise the software and the basis is to make
sure the software operates the way you would want it to in an everyday scenario. It is common
that Normal Range test procedures have in fact already been covered by the functional tests, as
functional tests cover everything. There may be additions made to cover aspects, if any, not
covered by the functional test. This ensures everything within the ‘Normal Range’ of operation
has been covered.
Going a step further, Robustness testing involves ‘invented’ scenarios and really tests the
software’s ability to perform under abnormal and challenging conditions. For example an event
that, albeit would occur infrequently but in reality could still occur during real operation, is
created to push the software to its absolute limits. In comparison to a physical structure, it would
be tested until broken to observe the maximum load it can take. Effectively, this test tries to
‘break’ the software. It includes an error injection principle which states; ‘Things that will not
occur under normal operating conditions.’ This can include errors in the initial input of
information, data corruption, system interference, and ultimately any possible scenarios,
including those so extreme that they really should never be reached. It is effectively a test
procedure covering every ‘What if?’
24. ~ 23 ~
Finally, Structural Coverage testing is ‘an analysis of how well you have written the
requirements and tested them.’ It is effectively ‘the final defence against software errors’ or the
‘exit criteria.’ In other words, it is more of an analytical procedure than an actual test. The
following section, which begins overleaf, will discuss the integration of a voice recognition
system with regards to requirements, safety and testing.
A testing procedure for voice recognition software can be found on page 29. This is the
test that was performed when testing the software to provide the results obtained within the
Appendices.
25. ~ 24 ~
Integration of Voice Recognition
Software
The Requirement
The flight deck of an aircraft often seems an array of endless switches, knobs, buttons
and screens displaying limitless amounts of in-depth information. It must be very demanding for
a pilot to try and troubleshoot any problems that may occur during a flight, or indeed to check all
aspects of aircraft configuration at a particular time, even more so under pressure during take-off,
landing or mid-air incident. Therefore, it would be thought that integrating voice recognition and
response software could ease the pressure on pilots allowing them to be in as relaxed a state as
possible during long transatlantic flights or even during pilot training.
Safety Assessment
It would be difficult to perform a full scale FHA and somewhat impractical, therefore a
sample table is shown below in Figure 17.
Function Failure
Condition
Phase Effect Class Verification
Pitch Nose Up Aircraft Pitches
up too early upon
take-off
Take-off Aircraft pitches
up at too low a
speed and stalls
at low altitude
Level A
Autopilot
Voice Control
Activation
The system is
activated when
not requested and
performs
unwanted
operations
Any
phase of
flight
Loss of control of
the aircraft at that
particular phase
of flight
Varies,
dependent
upon the
phase of
flight in
which it
occurs
Figure 17: Sample FHA Table for Voice Response Software
26. ~ 25 ~
With regards to Figure 17 above, an FHA is a very substantial piece of documentation
which would cover every possible consideration and would be passed amongst many parties to
include unthought-of scenarios and possibilities. Due to time limitations only a very brief sample
is shown above to cover the basic methodology in writing a Functional Hazard Assessment.
So, in terms of safety, every possible parameter must be considered and covered by
FHAs, FMEAs, and FMECAs etc. This report will only provide an understanding of the basic
background and explanation of these as, due to limitations with time and personnel, it would not
be realistic to cover every possible consideration.
Another method in which safety is the main consideration is fault trees. They consist of a
list of possible parameters that would either combine to cause failure, or independently cause the
same failure as some other occurrences. So, for the voice recognition principle, there are two
examples of occurrences in which there is unintentional operation of the system and then the
overall failure condition, with a fault tree analysis following.
Example 1: The first consideration is how the software can be safely and securely activated in
such a way that it only processes intended inputs, and does not act upon any vocal occurrences
during a flight such as any casual or required conversation between the Captain, First Officer,
Cabin Crew and ATC (Air Traffic Control) etc. The problem here, therefore, is unintentional
activation via an unintended vocal input. Hence, the system could become very unpredictable in
that the vocal input acted upon could be virtually anything said during the flight, including
casual, non-flight related chat, dependent upon words spoken and heard by the system. An
unrealistic, emphasised example, for the purpose of explanation could be the First Officer saying
to the Captain, or vice versa, ‘I almost accidentally turned the autopilot off (or on) there!’ Now
assuming that ‘Autopilot Off’ or ‘Autopilot On’ are both commands, the system could interact
and actually turn the autopilot off or on at an undesirable time, which was the initial avoidance
made by the First Officer in this instance. It is due to this unpredictability that the Criticality
Level of such an occurrence could be variable dependent upon the phase of flight. I would
therefore class this problem as Level B: Hazardous/ Severe.
27. ~ 26 ~
Example 2: Secondly, another problem occurrence could be the effects of Electro-Magnetic
Interference. At any one time, an aircraft has many, many signals and currents passing through
the extensive wiring system. With the operation of other systems such as the engines, APU
(Auxiliary Power Unit), power generators, radio signals etc. Electro-Magnetic Interference
would be a highly possible occurrence. This interference occurrence within the wiring system
could cause problems with performance within the system, and indeed any system causing
system failure, false inputs and/or corrupt data. Taking the same problem occurrence as above
into account – unintentional activation or deactivation of the autopilot – then ultimately the result
would be the same failure condition. There is a requirement for all wires within the system to be
there, and they are grouped together efficiently in terms of structural concepts and therefore this
problem could also be classed as Level B: Hazardous/ Severe dependent upon the interference
and failure condition.
28. ~ 27 ~
If both of these occurrences are taken into account as taking place during a phase of flight
in which the pilots have a very limited timeframe, for example around 20-30 seconds, in which
to react to save the aircraft from a condition of inevitable, catastrophic loss of aircraft and
occupants, the severity of occurrence is made clearer. So, should either of these conditions occur
and turn the autopilot on unintentionally and the aircraft performs a manoeuvre in which the
result would be catastrophic loss of the aircraft and its occupants if uncorrected within a period
of around 20-30 seconds, it would be presumed that the pilots would seek to quickly deactivate
the system and perform corrective action manually to regain control of and ultimately save the
aircraft. However, in terms of the occurrence of Example 1, imagine the aircraft does not respond
to the ‘Autopilot Off’ command. Human factors considerations would suggest that the pilot
would have raised adrenaline levels due to stress (see page 28) , and would not use the same
vocal tone as under normal conditions, therefore would there be sufficient time to manually
disable the autopilot and regain control? In terms of Example 2, imagine the same problem
occurs, this time however, the autopilot is unintentionally activated as a result of Electro-
Magnetic Interference. It would therefore be assumed that the pilots would react in the same
manner, as the same problem has occurred. If the system fails to react then would there be
sufficient time to manually disable the autopilot and regain control? Although, imagine the
software does respond, to the ‘Autopilot Off!’ command this time, but the Electro-Magnetic
Interference again almost instantaneously re-activates the system due to a fault in the wiring
system, is there any way of the pilots overcoming such a problem at all, never mind within in the
20-30 second timeframe?
29. ~ 28 ~
Thus, for the above conditions, a very brief fault tree is shown below in Figure 18:
Figure 18: Fault Tree Analysis
For the fault tree analysis in Figure 18 above:
1. Aircraft receives false input and acts upon an unintentional command
2. System does not have the ability to be overridden with vocal changes due to
adrenaline
3. Electro-Magnetic Interference occurs
4. There is no kill switch/ easy access circuit breaker to disable current within
the wires activating the system
5. AND Gates
6. OR Gate
In Figure 18, it can be seen that if occurrences 1 and 2 were to occur, or if occurrences 3
and 4 were to occur, then the result would be the loss of the aircraft and its occupants.
Therefore it would be thought that an efficient method of enablement, control and
override would be compiled to resolve this.
LOSS OF AIRCRAFT AND
OCCUPANTS
1. 2. 3. 4.
5. 5.
6.
30. ~ 29 ~
System Design
Taking the example fault tree problem occurrence in Figure 18, there would be a
requirement to redesign the software to overcome the problem occurrences discussed. For this
example, it can be concluded that, in order to maintain aircraft safety and preserve life in this
situation, then there would be a requirement for changes made to the enablement logic, control
logic and override logic of the voice recognition system to ensure that the pilot can activate,
control and deactivate the system quickly and effectively. In order to achieve this, a secure
enabling method, efficient control method and effectual override functions are established.
Enablement: It would be initially thought that until proven that the voice recognition system is
effective, it should use a PTT (Push-To-Talk) system. This would involve the pilot having to
physically push and hold a button whilst giving a vocal command to activate the listening device.
This would, significantly reduce the occurrence of factor 1 considered in Figure 18 as the pilot
would tend not to give a false input whilst holding in the required activation button. However, if
proven that the voice recognition system is of a very high standard then it would develop into a
totally vocal dependent system, independent of any physical input. There would be the
requirement however, for a vocal activation security code word or sequence in order to
differentiate an intentional command from any other vocal noise during a flight such as ATC or
discussion between Flight Crew and Cabin Crew. The vocal activation security code word or
sequence would essentially ‘grab the system’s attention’.
Control: In the early stages of the development process, it would be thought that the
system would be initially speaker independent (see Figure 19). However, as the system is
developed and inevitably establishes a higher accuracy, it would be thought that a speaker
dependent (see Figure 19) system would be incorporated. However, this presents its own new
difficulties. The sheer number of pilots that fly one specific aircraft would present problems with
regards to how the system stores and remembers the different voices. Therefore, an effective
method of training the system would be desirable. One suggestion could be that the system
listens to and learns a pilot’s voice during a pre-flight checklist, which would not waste time as
the pre-flight operations will also be completed during this.
31. ~ 30 ~
System Type System Type Description
Speaker Independent Functions regardless of the operator. This type of system is the
most favoured for everyday general usage. There is a decrease in
the accuracy rate and response rate therefore is not as accurate or
responsive as a Speaker Dependent system
Speaker Dependent Functions dependent upon the operator. This type of system is
required to be trained by the intended operator by detecting the
vocal properties, acoustic properties and speech patterns of that
particular speaker. There would be an increased response rate
and accuracy rate than the Speaker Independent system as a
result.
Figure 19: Speaker Independent and Speaker Dependent Systems
Override: It would be desirable for the system to feature an effective override method i.e. a
kill-switch, in the event of an emergency or system failure. In terms of a PTT system, the process
would be simple, release pressure from the button and the system stops listening, unless the
button becomes ‘stuck’. To overcome this, it could be thought that the button would be spring
loaded and not too tight against the orifice in which it is situated. When deactivated using this
method, if the aircraft is still performing a previous vocally activated manoeuvre or command,
then it could be overridden by the pilots following the procedures they would normally use in
overcoming that particular occurrence e.g. turn the autopilot off manually, if previously or
undesirably activated by voice. In terms of the Speaker Dependent system, a vocal command
would be used to deactivate the system. Although from a Human Factors point of view, stress
(see page 28) could cause changes to the vocal properties, acoustic properties and speech
patterns of that particular speaker which could lead to difficulties with vocal deactivation.
Therefore, it would be thought, that a kill-switch or circuit-break method would need to be
considered. This would allow the pilots to kill power to the system and regain normal control of
the aircraft whilst eliminating the Electro-Magnetic Interference problems discussed previous
due to the absence of current flowing through the wires.
32. ~ 31 ~
Human Factors – Stress
There are many Human Factors aspects which would affect the proposed Voice
Recognition operation, and every single one of these factors would need to be taken into deep
consideration and analysed in terms of safety. To provide an example of the Human Factors
Considerations aspect of integrating a Voice Recognition package, a very brief discussion of the
effects of stress will follow.
Stress is the reaction of a person to mental and physical strain placed upon him/her. This
response to stress includes the release of chemical hormones for example adrenaline, into the
bloodstream. The person’s metabolism is increased which provides positive energy to the
muscular system. As a result, factors including blood sugar, heart rate, breathing, blood pressure
and sweating are increased. The factor which causes stress is known as a stressor and these can
either be physically, mentally or physiologically induced. An example of physical inducement
would be g-force or vibration. An example of mental inducement would be working with
difficult personal circumstances out-with the work environment. Finally an example of
physiological stress would be fatigue.
Stress can also be divided into two types, chronic and acute. It would be assumed that if a
pilot were experiencing/ diagnosed with chronic stress then he/she would be deemed unfit to take
control of an aircraft, therefore acute stress is more relevant in this instance. Acute stress is
induced by a sudden sense of danger. It triggers a ‘do or die’ scenario that is either real or falsely
perceived/exaggerated. It would be normal for a person to be able to effectively deal with acute
stress at that time, however if it is on-going it could become chronic.
In terms of the Voice Recognition system, it would be thought that any signs of excessive
mental or physical stress would have an effect on the competency of the pilot and his vocal
rhythms could be altered by this, and therefore the performance of the system would also be
affected as an incorrect input rarely achieves a correct output once it has been processed.
Examples of physical stress include tension in the muscles, shivers and the clench of the jaw,
similar to the phenomena experienced when on a thrill ride at a theme park for example. It would
be thought that any vocal input under these conditions would have a significantly reduced
accuracy and hence the operation of the system would suffer.
33. ~ 32 ~
Test Procedure - Methodology
Job Setup
Step Procedure Completed
(Tick)
1 Activate the power supply for the intended equipment
2 Perform the start-up procedure for the computer in which the software
package is installed
3 To activate Windows Speech Recognition follow these steps:
- Click ‘Start’
- Click ‘All Programs’
- Click ‘Accessories’
- Click ‘Ease of Access’
- Click ‘Windows Speech
Recognition’
4 Follow the on-screen instructions to check that the microphone intended for
use is plugged in and operational
Normal Range Test – Action
Step Procedure Completed
(Tick)
1 Follow the ‘Job Setup’ procedure
2 Complete the speech recognition tutorial (if required)*
3 Open the Microsoft Word program using the following steps:
- Click ‘Start’
- Click ‘All Programs’
- Click ‘Microsoft Office’
- Click ‘Microsoft Word (2010 etc.)’
4 Ensure that the cursor is flashing, signalling that it is prepared for a typed, or
vocal input
5 To activate the listening device, have the test performer say ‘Start Listening’
and record how many attempts it takes for the system to respond properly
6 Have each test performer, in turn, say each of the four predetermined
sentences (see Figure 21) three times and record how the output compares to
the original sentences
7 Have the test performers deactivate the listening device by saying ‘Stop
Listening’ and record how many attempts it takes for the software to respond
8 Perform the ‘Close Up’ procedure
34. ~ 33 ~
* For the basis of comparison, during the first stage of Normal Range testing, the tutorial will
not be used. The second stage of the test will require the tutorial to be completed as the system is
trained to the voice of the test performer. The first and second tests will determine whether
training the system produces a higher percentage of accuracy than leaving the system un-trained
(see Figure 19).
Robustness Test – Action
Step Procedure Completed
(Tick)
1 Follow the ‘Job Setup’ procedure
2 Complete the speech recognition tutorial (if required, see previous)*
3 Open the Microsoft Word program using the following steps:
- Click ‘Start’
- Click ‘All Programs’
- Click ‘Microsoft Office’
- Click ‘Microsoft Word (2010 etc.)’
4 Ensure that the cursor is flashing, signalling that it is prepared for a typed, or
vocal input
5 Repeat the Normal Range Test – Action steps 5, 6 and 7,
incorporating the following factors:
- Background noise
- Unintentional command
- ATC transmission at the same time
- Quick speech
- Slow speech
- Shouting over increased
background noise
- Heightened stress
(Details/ideas of how to achieve these factors will follow the test procedure)
6 Perform the ‘Close Up’ procedure
35. ~ 34 ~
Close Up
Step Procedure Completed
(Tick)
1 Close Microsoft Word by either clicking the Red ‘X’ at the top right of the
window or by right-clicking the ‘Microsoft Word’ icon on the toolbar and
clicking ‘Close’. There is a requirement to save the document for compiling
results later
2 Close Windows Speech Recognition by either clicking the Red ‘X’ at the top
right of the window or by right-clicking the ‘Windows Speech Recognition’
icon on the toolbar and clicking ‘Close’
3 Disconnect the microphone used
4 Perform the shut-down procedure for the computer used during testing
5 Disconnect the power supply to all equipment
Why and How to create Factors Affecting System Performance
Background Noise
The flight deck is by no means a quiet environment, after all aircraft can be travelling
through the air at speeds of up to and even beyond 500mph. To simulate the noise which would
be heard due to air passing over the flight deck exterior in terms of the test procedure, the most
practical method would be to conduct the test with the Flight Simulator program in operation,
with an aircraft in flight creating the appropriate simulated noise. The test would then be
conducted with the performer and microphone placed close to the output noise of the Flight
Simulator program.
Unintentional Command
There is the potential for the pilots to be unaware of the software status i.e. whether or
not the listening device is active. An example of a possible problem could be the Captain
briefing the First Officer of an upcoming flight procedure. The software could receive and
process this input and perform an unwanted manoeuvre too early. A sample sentence could be:
‘Keep an eye on airspeed, and remember, don’t bring the landing gear down too early!’
36. ~ 35 ~
If processed as a command, the aircraft could bring the landing gear down and significant
structural damage could result. It would be thought however, that the pilots would be purposeful
in giving commands to the system and that their vocal tones would differ between giving
commands and general chat. This factor would then measure the sensitivity of the device.
ATC Transmission
Similar to the two previously discussed factors in that background noise in the form of an
ATC transmission to another aircraft could be picked up from the pilot’s earpiece as an
unintentional command by the system. So, for this part of the test, there would be a requirement
for two test performers, one acting as the pilot and one as ATC. The pilot would say the
predetermined sentence as outlined in the test and the ATC would say another command as if
talking to another aircraft, at the same time. This test is to see which, if any, command the
system follows or whether the system becomes confused.
Quick/Slowed Speech
Due to the effects of hypoxia or adrenaline, the voice of the pilot could be quickened or
slowed, producing an abnormal input to the system. This would be simulated by the test
performer speeding up/ slowing down their speech accordingly. This further tests the accuracy
and sensitivity of the system under abnormal conditions.
Shouting Over Increased Background Noise
Follows the same idea as background noise outlined previous although would involve
significantly increased audio output. This would simulate a situation in which the aircraft is in a
state of real danger and the pilots could be frantically trying to regain control of the aircraft. This
could be achieved by increasing the volume of the Flight Simulator program, having aircraft
warning systems going off and the test performers shouting over this in an attempt to process the
commands.
37. ~ 36 ~
Heightened Stress
This would be to simulate a period of heightened stress (see page 28) for the pilots and
could be simulated via some form of physical test to induce breathlessness and a sign of
heightened stress. Simulation could be achieved by:
- Treadmill (ideally)
- Holding/ lifting a heavy object
- Arm wrestle
- Shadow boxing
- Star jumps
Next, a set of sentences and the test performers should be obtained. Now, ideally, this test
should be performed with a very high number of test performers from a very wide variety of
backgrounds and a wide range of different sentences and possible vocal inputs, but due to time
limitations and the conditions for the writing of this report, it would not be realistic to do so,
therefore the test will be performed with 4 sentences by 5 people.
Participant Participant Name
1 Frazer Hamilton
2 Kirsten Gallagher
3 Antony Templeman
4 Eric Mutasa
5 Gordon Keary
Figure 20: Test Performers
38. ~ 37 ~
Statement Number Test Sentences
1 She sells sea shells on the seashore
2 Turn left heading 270
3 Full throttle
4 Climb flight level 360
Figure 21: Test Pre-determined Sentences
The results are then compiled into spread-sheets which feature as Appendices in this report.
Test Procedure - Summary of Findings
Before outlining and summarising the results of the test, it should be noted that
abnormalities are to be expected within the results. This is due to time limitations experienced
when performing the test involving Eric Mutasa. This test performer was only available to
perform the Normal Range tests and did not complete the Robustness section of the test.
Figure 22: Table of results
0
10
20
30
40
50
60
Statement 1 Statement 2 Statement 3 Statement 4
Statement
Average Percentage Accuracy
Average Percentage
Accuracy
39. ~ 38 ~
It can be seen in Figure 22, that Statement 1 clearly demonstrates the highest accuracy
level at 55%. This demonstrates that the software could understand and process this sentence
easier and could be symbolic of the system finding longer, flowing sentences easier to process.
There seems to be an alarming level of accuracy with regards to Statement 3 at just over 10%.
This could suggest that the system struggled with the much shorter, to the point, statement.
Statements 2 and 4 are of a similar accuracy and are essentially the midpoint in terms of length
of the sentences and accuracy. However, the more words there are the more accuracy there is
likely to be, so the results are typical of what would be expected in this case.
Looking at Appendix A and Appendix B, it can be seen that there is a very wide variety
of accuracy levels within every aspect of the test. There is no real consistency between the type
of test, and the test performers, and the results seem somewhat random. Although, in Appendix
C, with the exception of Participant 4, the results for Statements 1 and 2 show the only real signs
of consistency in terms of the statements. In terms of the Participants, the majority of the
participants follow similar performances relating to the graph in Figure 22 above, in that they
performed best in Statement 1, showed significant decrease in Statement 2, further decrease in
Statement 3 and an increase in Statement 4. Participants 1, 3, and 5 have accuracy levels of
around the same general area within 10% in all cases. This was expected as these performers
were all males of British origin with a mixture of Scottish and English accents. Participant 2
showed the same pattern but a significantly lower accuracy and was a female of British origin
with a Scottish accent. The reasons for the abnormalities with regard to Participant 4 were
discussed earlier, and it should be noted that he was also a male of British origin with an English
accent. Although during the Normal Range tests he performed, it can be seen from Appendix C
that he produced accuracies of a similar trend to that of Participants 1, 3 and 5.
As also discussed previous, this test would have been conducted with many more
participants from all over the world to rigorously test the software and give more accurate
results, but due to time limitations and the basis of completion of this report, a rough and brief
test was conducted to show the theory behind a test procedure to enhance the reader’s
understanding.
40. ~ 39 ~
Flight Deck Evolution – The Path to
Automation and Beyond
A Conclusion
Technology within the aviation industry has come a long way, and it can be clearly
observed that the flight deck of an aircraft is certainly at the forefront of modernisation and
increased automation used on board the most modern aircraft nowadays. There are limitations
however in what can be integrated; even the most proven technically advanced systems are put
through a rigorous safety assessment, certification and testing process before being seriously
considered for integration. Safety is top priority in aviation as any system failure could
potentially put the lives of many people at risk both aboard the aircraft, and on the ground. It is
thus that every possible outcome that can occur is assessed, analysed and considered in terms of
certification, safety assessment and incorporates the likes of human factors (stress etc.) and
failure modes. It is thus that, in terms of Voice Recognition Software, it can be concluded that it
is not a practical system for integration based on the results of the test carried out in this
particular report. The results in Figure 22 highlight that the system is not accurate enough and
there are too many uncertainties and erroneous outputs from the system for it to control an
aircraft. The system would need to be refined and updated to incorporate differing accents, inputs
and be somewhat resistant to background noise. Careful consideration would need to be given to
every possible variable factor within the system, its inputs and outputs to ensure that the system
operates accurately and safely. Military examples of voice recognition can be seen aboard the
Typhoon fighter aircraft, therefore voice recognition is certainly a valid and realistic system to be
considered for use aboard commercial aircraft. However, at this time, it remains simply a
concept, that could one day control the commercial aircraft of the future.
41. ~ 40 ~
References
Websites and PDFs
Author unknown (2007) Aeronautics - Flight Instruments - Level 1,
http://www.allstar.fiu.edu/aero/fltmidinst.htm, Date accessed 12/10/2011
Author unknown (Date unknown) Instruments – General,
http://www.nordian.net/pdf/easa_instrumentation_demo.pdf, Date accessed 19/10/2011
Author unknown (Date unknown) Aircraft System Test Procedures Preparation & Release,
http://www.nasa.gov/centers/dryden/pdf/87455main_DCP-O-011.pdf, Date accessed 23/11/2011
Kelly, T.P and Wilkinson, P.J (Date unknown) Functional Hazard Analysis for Highly Integrated
Aerospace Systems, http://www-users.cs.york.ac.uk/~tpk/ieefha.pdf, Date accessed 02/02/2012
Author unknown (2010) Standard glossary of terms used in Software Testing,
http://www.astqb.org/educational-resources/glossary.php, Date accessed 02/02/2012
Various Authors (2009) Notice of Proposed Amendment (NPA) No 2009-12,
http://www.easa.eu.int/rulemaking/docs/npa/2009/NPA%202009-12.pdf, Date accessed
04/03/2012
Author unknown (2005) FAA Federal Aviation Regulations (FARS, 14 CFR),
http://www.flightsimaviation.com/data/FARS/part_25-1301.html, Date accessed 04/03/2012
Author unknown (1996) AC-251309 System Design and Analysis, http://www.astech-
engineering.com/systems/avionics/aircraft/faaac251309a.html, Date accessed 04/03/2012
Author unknown (2011) AN-656 Understanding the Operation of a CRT Monitor,
http://www.ti.com/lit/an/snla017/snla017.pdf, Date accessed 04/03/2012
Haddon, D.R and Whittaker, C.J (2002) Aircraft Airworthiness Certification Standards for Civil
UAVs, http://www.caa.co.uk/docs/1995/srg_acp_00016-01-120203.pdf, Date accessed
02/02/2012
Hilderman, V (2011) Do 178-B to Do 178-C Impact on Avionics Verification and Certification,
https://www.signup4.net/Upload/BOOZ14A/SAFE23E/Day1_03_MilSymposiumJune2011.pdf,
Date accessed 13/04/2012
Various authors (2006) Human Factors and Aviation Medicine, http://flightsafety.org/hf/hf_jan-
feb06.pdf, Date accessed 16/04/2012
42. ~ 41 ~
Books
Pallett, E.H.J (1992) Aircraft Instruments & Integrated Systems, New Jersey: Prentice Hall
Baghai, T and Hilderman, V (2007) Avionics Certification: A Complete Guide to DO-178
(Software), DO-254 (Hardware), Leesburg: Avionics Communications Incorporated
Various Authors (2008) Pilot’s Handbook of Aeronautical Knowledge, Oklahoma: United States
Department of Transportation, Federal Aviation Administration, Airman Testing Standards
Branch
Various Authors (2000) FAA System Safety Handbook, United States of America: Federal
Aviation Administration
Various Authors (1987) Fault Tree Handbook, Washington D.C: Nuclear Regulatory
Commission
Various Authors (2001) Avionics Handbook, Washington D.C: CRC Press
Keary, G (Date unknown) Test Procedure, Prestwick
