1. How to Maintain Flight Testing in Human
Spaceflight Programs
PM Challenge
February 9-10, 2011
Robert Ess Jay Estes
NASA – Johnson Space Center
2. Outline
• Importance of FT in Human development
• PA-1 Highlights
• Ares I-X Highlights
• Common Themes between PA-1 and I-X
• Challenges to Flight Testing
• Conclusions
3. Importance of Flight Testing for
Development of Human-Rated Spacecraft
• Failure of ships ferrying humans to/from earth must be virtually non-
existent
• Ground testing cannot replicate stacked-environs for integrated
systems well
– If available it is very expensive
– Usually only simulated effects for many aspects
– Much easier to do verification testing (corners of the box), Flight test is
about validation (all systems work together to perform a selected
trajectory/mission
• Component testing on ground cannot eliminate integrated system
“blind-spots”
– Ex. Shuttle foam debris, Hubble aberration
• Development of complex systems is spread across many suppliers,
with independent functions
– Operate with different mind-sets, different ways of thinking (coordinate
frames, analysis techniques, uncertainty factors, testing rigor)
– Interfacing of these systems always has inherent risks, and potential
blind-spots (ICDs and IRDs wont catch everything)
• Integrated testing of full-scale systems in the right environment is the
most-valuable kind of design validation that exists - proof is in the
performance
4. Development Flight Testing
• “Early” flight tests to assure that Engineering team is on the right track and to
anchor analyses and tests
• Often planned at beginning of a Program/Project
• Often deleted due to budget and schedule challenges
– Budget challenges can be driven by “expected full rigor” from your discipline teams
• One of the most significant benefits is the entire production machine (production
“throughput”, procurement, drawing release, configuration control, analysis and
test documentation, quality records, requirements validation, safety processes,
waiver and deviation processing, flight readiness review rigor/processes, etc)
– ALL project systems become functional (some through trial and error) usable infrastructure for
the “real project” ahead
• Engages a core team, develops critical team member buy-in, and motivates team
members far greater than “paper projects”
– A dedicated flight test teams can become deeply “invested” in the project goal
– Personal rewards for team far greater than “book management” and “interface and
requirements management”
– Provides a path to leadership development within the team, and a testing ground for
management decision-making processes
2011 PM Challenge 4
5. National Aeronautics and Space Administration
Crew Exploration Vehicle
Progress and Success
Jay Estes
www.nasa.gov
6. Pad Abort 1
Overview and Results
• Test of a newly developed, full scale Launch abort system for Orion
– Protect crew by providing escape system from danger at pad, or off-nominal ascent
– Test data used to correlate models and refine analysis targets
– Demonstrated the performance of three new types of motors, and innovations to lessen
their weight and eliminate need for ballast of a passive system
• Summary Primary Test Objectives Demonstrate
– Performance of the Launch Abort System (LAS) and LAS/crew module interface
– Capability of the LAS to propel the module to a safe distance from a launch vehicle
– Stability and control characteristics of the LAV in the flight environment
– Determines the performance of the abort, jettison and attitude control motors
– Demonstrates abort event sequencing from abort initiation through LAS jettison
– Obtains LAS/crew module interface structural loads and external acoustics data
• Secondary Test Objectives Demonstrate
– Parachute assembly system event sequencing
– Performance of the main parachute systems
8. What will be seen
Time Event
1. 0.00 AM/ACM ignition
2. 6.45 AM burnout
3. 10.05 Begin re-orientation
4. 15.77 End re-orientation
5. 21.03 LAS Jettison
6. 22.02 FBC jettison
7. 24.56 Drogue mortar fire
8. 30.56 Pilot mortar fire
9. 49.32 LAS touchdown
10. 52.54 Reach 33 ft per sec
descent rate
11. 96.83 CM touchdown
9. Crew Module on Pad
National Aeronautics and Space Administration
12. Orion’s First Flight Test
Forward Bay Cover Drogue chute Pilot chute Main Chute
Jettison (23 ft dia. each) deploy of mains (116ft dia. Each)
Forward bay cover
jettison
National Aeronautics and Space Administration
13. PA1 Results and Findings
• PA1 was completely successful!
– Performance on virtually all systems was nearly flawless
– Findings indicate the system went a little higher, faster, farther and
landed softer than expected
– Complex reorientation maneuver completed faster than expected
– LAS re-contact at jettison was non-existent – very clean separation
– Forward bay cover jettison was shown to have no chance of
entanglement with drogue chute deployment
– LAS jettison plumes had no discernable affect on chutes
– LAS successfully propelled the vehicle to sufficient altitude to deploy
main chutes to full open was unfounded (more than 70 sec on full-
open mains)
14. PA1 Results and Findings
• CM landing achieved 23 ft/s vertical decent rate, all avionics remained on
at landing despite not being designed to survive ground impact, no
structural damage
• CM suffered only minor cosmetic (paint) external blemishes due to the
flight. Reverse flow abort motor nozzles provide excellent separation of
abort plumes from CM
• Dual string avionics was never needed, no software or hardware failures
or glitches during the flight
– Identification of the need for “smart release” from drogues on PA1 has lead to
the planned incorporation of this logic on the manned vehicle
• Both sides of redundant pyrotechnic chains fired on all components, no
malfunctions, all fired on-time as expected.
• Less than 1% of instrumentation (692 sensors) were suspect at launch,
virtually no failures in flight
15. Examples of Challenges
for PA1
• Commit to Flight Readiness
– Attempts to redefine and simplify the process for flight tests met with great
resistance
– Endorsements only come with a heavy burden of proof via ground test,
analysis, and part certification
• Discipline teams didn’t have any solid guidance from management on
“how to back off” from human spaceflight rigor
– Most disciplines applied full-rigor due to no definition of “what to back-off to”
• Org-to-Org consistency on handling of “flight hardware”
– 2 categories: flight hardware (full-rigor), or “other hardware” (deserves no
paper or special care)
– Need a middle ground where projects can provide sufficient guidelines for HW
processing (selectable, based on guidance from project)
16. Examples of Challenges
for PA1
• Interfacing multiple centers and organizations can drive multiple layers of
“responsible parties”
– Ex: ground safety at WSMR (NASA + Army + Lockheed)
• Certification testing durations: Abort condition is extreme (170dB
acoustic, high energy random vibe, but only lasts 3-4 seconds)
– Standards for pyrotechnic/ordnance components: +6dB for at least 3mins
– Sometimes, these levels are unachievable in ground testing, and include
significant dwell time to ramp test rig up desired levels
– Components fail due to excessive exposure (fatigue) in attempts to certify to
recognized standards
18. Ares I-X: The Basics
• Mission overview
– Uncrewed, suborbital development flight test
– Collected engineering data from launch to first stage recovery
– Support Ares I critical design review
– Early data for development – “Engineers Flight test”
• Launch operations
– Vehicle launched 11:30 a.m. Eastern Time, October 28, 2009, from Launch Complex
39B at Kennedy Space Center (KSC)
• Hardware overview
– Primary flight hardware consisted of a four-segment solid rocket booster from the
Space Shuttle program
– Rocket controlled by Atlas V rocket avionics
– Repackaged Peacekeeper Missile 4th Stage propulsion for Roll Control
– Simulated flight hardware for Upper Stage and Crew Module
20. Ares I-X Launch
Was Successful!
Demonstrated control of a dynamically
similar, integrated Ares I/Orion, using
Ares I relevant ascent control algorithms
Performed an in-flight separation/staging
event between a Ares I-similar First Stage
and a representative Upper Stage
Demonstrated assembly and recovery of a
new Ares I-like First Stage element at KSC
Demonstrated First Stage separation
sequencing, and quantified First Stage
atmospheric entry dynamics, parachute
performance
Characterized magnitude of integrated
vehicle roll torque throughout First Stage
flight
22. Ares I-X Roll Out
National Aeronautics and Space Administration
23. Ares I-X
First Stage Recovery
National Aeronautics and Space Administration
24. Examples of I-X Challenges
1. Challenge to “contain” analyses to
desired scope
2. Difficult to communicate what “Take
more risk” really means
-So some disciplines used usual human spaceflight rigor
3. CoFR process grew in scope as got
closer to flight
-Additional reviews and “angst” as to what is enough
4. Centers had own standards and own
review/approval processes for logistics,
pyros, FS
24
Picture taken by
Calvin Turzillo
25. Common Themes Between
PA-1 and Ares I-X
• Both tests were early in the design cycle:
– I-X was well before Ares I launch vehicle CDR
– PA-1 occurred approximately 4 months before Orion PDR
• Both test configurations combined items that directly apply to final config
and are “one off”s
– I-X used LV OML and flight control algorithms etc
– PA-1 used early adapter cone LAS/CM attachment, boilerplate crew module
simulator, and “protoflight” LAS HW
• Both tests had many dedicated sensors
– Both had debates with technical community on what is required vs desired –
there are never enough data systems onboard
• Both were very successful and provided huge amounts of data to
engineering team and integrated team (logistics, CM, requirements…)
– Data is still being applied in model correlation
• Both were attempted to be done in a more efficient, less Human
Spaceflight rigor and were met with some impediments
26. Common Themes Between
PA-1 and Ares I-X
• Both were attempted to be done in a more efficient, less Human
Spaceflight rigor and were met with some impediments:
– Risk acceptance (which happens at the project manager level) was not easily translated
down to engineering level
– CoFR process encourages questions of “Have we done everything we possibly could
do?”
• Answer should be “No” for early flight tests
• Institutional managers asked to sign CoFR when they don’t necessarily agree with risk acceptance level of
Project Manager (torn between programmatic requirements and signing their name that system will work)
– Default approach was to start with current standards/specs and reduce on a case by
case basis
• Not practical to start with Human spaceflight rigor and whittle down
• Need to start with clean sheet and build up as justified
• Again, related to risk acceptance. Project management doesn’t see every analysis or test
• Perception of importance and need for success increased dramatically as
launch date approached
– NASA is very image-sensitive and any failure (even a good one) is frowned upon by many
in and outside of NASA
27. Point and Counterpoint for Early Flight Testing
• Development flight tests do not need to be the final config. Emphasis is on
the key areas that are being tested and the end-to-end execution. The rest
can be ‘simulators’ etc
– COUNTER: “One-off” h/w and s/w distracts the team from the final
configuration. Extra Money is spent on a unique flight test configuration much
of which does not apply to final config.
• Flight tests can just focus on key critical data items that cannot be obtained
in any other manner, don’t need final configuration, final design etc
– COUNTER: Need to get as much data as possible from the flight. Need as many
sensors as possible and claim as many test objectives as possible.
28. How Far Do You Go?
• When does an unmanned flight test on an isolated range need redundant
systems?
– Is the cost of buying, integrating, testing, and operating two sets of
components less than a reflight?
• Is use of full-rigor on systems requirements, and verifications per NASA
procedural requirements required for unmanned flight test?
– No. Ideally any ‘relief’ on such requirements should be documented up front. But flight
tests tend to be very “fast track” and start up quickly and typically don’t spend months
documenting relief
– A solution: Anoint a dedicated manager with authority to decided and accept risk using
Technical Authority to illuminate risks.
• Ares I-X and PA1 were fully successful as very first full scale flight test, a
testament to procedural rigor across systems, discipline and their
integration
– But at what cost? If you can reduce rigor and fly with more risk at half cost –
opportunity for cost savings or additional tests emerge, with no loss in
objectives
29. Risk vs Success
• Approval processes and risk acceptance tend to mirror planned processes
for Human spaceflight
• Tug of war between timely (i.e. get it soon) flights and risk tolerance
– Default levels honed from decades of human spaceflight
• What is true Objective of test?
– Instrumentation
• “As much as possible” vs “just enough”
– Flight Demonstration
– OR: establishment and exercise of project production processes, approvals,
and mechanized infrastructure
30. Items to Be Addressed for
Future Flight Testing
• If we are performing developmental flight tests and failures occur (they
will) – why should mishap boards be convened?
– How is the occurrence of a failure of a new craft, in a heretofore un-
experienced environment considered something we “must correct and
avoid”? (Write Mishap Plans carefully to allow failures of “unit under test”)
– Failure in flight tests should be expected, embraced. Failures always lead to
learning
• Inter-discipline protection of margin
– Discipline teams are trained to protect sufficient margin to “guarantee”
performance
– Stacking of margins from discipline to discipline drives design conditions which
become very hard to satisfy with the fully integrated system
– Flight test vehicle development should clearly define goals, provide “back-off”
from crewed-systems standards including both test levels and durations for
acceptance and certification for flight test. (e.g. 3-sigma is not a requirement)
– Project Manager (and Chief Engineer) should “own” the integrated margin
• CE needs to be cognizant of summed margins across system and manage on an integrated
system approach, not system by system
31. Items to Be Addressed for
Future Flight Testing
• Lean toward executing on schedule as opposed to executing without
failure
– “Better” is the enemy of “good enough” (and “perfect” is the enemy of
“better”!)
– Highly trained and motivated teams have a difficult time accepting non-
optimal solutions (Project Manager and TA’s need to be on the same page and
provide frequent and clear guidance)
• Testing rigor tends to be driven by standards which are “tailorable” but
without guidance, disciplines default to full-rigor, highly conservative
approaches
– Management owes the team clear guidance on what can be allowed for flight
test –vs- what must be done for crewed vehicles
32. Other Suggestions for Future
Human Flight Test Programs
• Start Early with a configuration sufficient to test a few critical objectives
– Every test does not need to be Prototype
– The one-off ‘overhead’ will still be very useful to training the team for more complex
configurations
• Limit Scope of early flight tests
– Risk-based (“Top Down”) approach to deriving objectives as opposed to “bottoms up”
approach a la wish list
– Use project/program risk database to identify key areas OR improve risk databases
• Don’t overestimate the relevant experience of your team and downplay or
delete need for early flight tests
– Not just technical team, but CM, logistics, COFR processes
– Flight h/w will be handled differently than a ‘pathfinder’ so the whole team will learn
from a development flight test
– Go find a few people with development flight test experience
• Consider “spiral vehicle development” starting with simple, lo-fidelity test
articles, and increasing rigor, capability, objectives, and test conditions
through a few incrementally directed flight tests
– Include mock-ups and pathfinders as the crudest forms
– Develop a useful series of vehicles with incremental maturity
33. Conclusions
• Flight Testing in Human Spaceflight is critical for new
spaceflight programs/projects
• Tendency is to “price” test out of feasibility due to
Human Spaceflight Rigor being used as default
content for all aspects
• Risk Tolerance and acceptance drives cost and
schedule
– Challenging to delegate “Take more risk”
– Need to translate into relief on standards, review etc
• NASA team needs benefits from real flight tests
– All disciplines can benefit
2011 PM Challenge 33
Conducted at the Launch Complex 32E at the U.S. Army’s White Sands Missile Range in New Mexico The Pad Abort 1 test will be the first full-scale test flight demonstration of the Launch Abort System (LAS) system using a single use mock-up crew module. The test will enable engineers to verify the launch abort system can safely execute an abort and recovery sequence during pre-launch countdown and the critical phases of an ascent abort The LAS abort motor will produce over a half-million pounds of thrust within milliseconds to pull the Orion mockup from the platform The motor will burn for approximately six seconds, with the highest impulse and velocity (600 ft/sec) in the first 2.5 secondsIt will safely propel the test vehicle upward approximately one mile from the launch pad. The LAS and crew module will separate, parachutes will deploy and the crew module will land approximately one mile downrange from the launch pad after about 100 seconds aloftThe LAS consists of three unique propulsion systems Reverse-flow Abort Motor to pull the crew module safely away in a contingency during launch and ascent Attitude Control Motor that provides directional control of the system during abortJettison Motor that separates the LAS from the Crew Module after the abort, and during a nominal missionCombining direct observation, experience, and physical measurements with analysis leads to deeper understanding and insight to:Learn as much as possible, as early as possible in the product development cycleAnchor models and engineering tools with actual flight performance dataGain a deeper understanding of the vehicleValidate the initial vehicle designAcquire early design, manufacturing, integration and operations experience with a mature prototype vehicle Observe subtle and unexpected problems
Attitude Control Motor (ATK/Elkton)Jettison Motor (Aerojet)Abort Motor (ATK/Bacchus)Crew Module (LaRC/DFRC)Nose Cone Assembly (Orbital)Canard Interstage (Orbital)Interstage (Orbital)Adapter Cone Assembly (Orbital)
Test Objectives:Test data will have wide applicability to future launch vehicles and will demonstrate the performance of three new types of motors, and innovations to lessen their weight and the need for ballast of this systemPrimary Test Objectives Demonstrate Performance of the Launch Abort System (LAS) and LAS/crew module interfaceCapability of the LAS to propel the module to a safe distance from a launch vehicleStability and control characteristics of a crew module in regards to the LASDetermines the performance of the abort, jettison and attitude control motorsDemonstrates abort event sequencing from abort initiation through LAS jettisonObtains LAS/crew module interface structural loads and external acoustics dataSecondary Test Objectives DemonstrateParachute assembly system event sequencingPerformance of the main parachute system.Criteria for Success:Fully Successful All above objectives achieved plus deployment of forward bay cover, 2 drogues, 3 pilots and 3 main chutes extract and inflate to first stage.Successful ACM continues firing and controlling as or nearly as expected and controls LAV downrange, conducts a successful reorientation, and delivers the entire LAV to the proper attitude for LAS jettison.Minimally successfulAbort motor and attitude control motor ignite and LAV (launch abort vehicle) achieves lift off with both motors firing.
Pad Abort 1 was the first in-flight demonstration of the Orion crew exploration vehicle’s launch abort system. The May 6, 2010 flight test at the U.S. Army’s White Sands Missile Range in New Mexico validated the launch abort system, demonstrated the performance of three new rocket motors and the parachute recovery system, and served as a design and development pathfinder for future crew escape systems. The launch abort system includes three new solid propellant motors, which all performed flawlessly during Pad Abort 1. During the flight test operations, the abort motor fired with approximately 500,000 pounds of thrust to drive the crew module from the pad; the attitude control motor fired simultaneously and provided the nearly 7,000 pounds of force required to maintain stability and vehicle trajectory, propelling the launch abort system to a height of approximately one mile; and the jettison motor separated the crew module from the launch abort system in preparation for parachute deployment.
Activated at ground level during the Pad Abort 1 test, Orion’s launch abort system propelled the crew module 500 feet in the first three seconds of flight. During the total 97 seconds of flight, the launch abort system reached an altitude of 5,000 feet, traveling at a maximum speed of 600 miles per hour.The abort motor propellant burns at a temperature of 4,456 degrees Fahrenheit, nearly half the temperature on the surface of the sun. While that temperature is hot enough to boil steel, the interior temperatures of the crew module measured only 75 degrees Fahrenheit during the Pad Abort 1 flight test.Combustion gases exiting the motor nozzles travel at a speed of 2,600 miles per hour, more than three times the speed of sound and two times the speed of a bullet shot from a rifle.Mission Timeline:Ignition = 0 s End reorientation = 16 sAbort motor burnout = 7 s LAS jettison = 21 s Begin reorientation = 10 s LAS touchdown = 49 s
At crew module separation, stabilizing drogue parachutes deployed, followed by the pilot parachutes and then the three main parachutes that facilitated the crew module’s nearly two-minute descent to the landing site, approximately one mile from the pad. Mission timeline (continued):Forward bay cover jettison = 22 s Drogue mortar fire = 25 s Pilot mortar fire = 31 s Slow to 33 ft/s decent = 53 s Crew Module touchdown = 99 s
In the Vehicle Assembly Building's High Bay 3, Super Stack 5 is secured to the Ares I-X segments already in place on the mobile launcher platform. The 327-foot-tall rocket is one of the largest processed in the bay, rivaling the height of the Apollo Program's 364-foot-tall Saturn V. Five super stacks make up the rocket's upper stage that is integrated with the four-segment solid rocket booster first stage.
The Ares I-X is the first major assessment of the crew launch vehicle.The objectives of the suborbital flight test include:Assessment of ground facilities and operations at KSCVerification of design effectivenessCritical data-gathering regarding in-flight safety and stabilityEvaluation of the 327-foot integrated stack, which includes a simulated crew module and Launch Abort System, during ascent
In the Atlantic Ocean off the coast of NASA's Kennedy Space Center in Florida, United Space Alliance Recovery Operations divers and personnel approach the vertically floating Ares I-X first stage following the launch of the flight test mission as the solid rocket booster recovery ship Freedom Star stands by to tow it back to the center.
