1. National Aeronautics Space Administration
Using a Systems Engineering Approach
to Develop NASA Engineering Talent
Mr. John Marinaro and Dr. James May
NASA Safety Center
www.nasa.gov

2. What does NASA’s future hold?
2

3. One Thing is Certain…
NASA faces an Engineering Challenge!
Of the 11,216 NASA Engineers, 60% will be eligible to retire in the next 10 years.
Of the remaining engineering population, 33% have less than five years of
experience. These statistics indicate that NASA will face an engineering
knowledge drain over the next decade.
This is not a Center-specific challenge, but an Agency and Leadership Challenge.
NASA Engineering Population
Eligible for
Retirement
<5 Yrs
Experience
>5 Years
Experience
3

4. The Data to Support the Challenge (2011)
NASA 08xx Series (All Centers)
50 to 70 or
ServiceAge Under 20 20 to 24 25 to 29 30 to 34 35 to 39 40 to 44 45 to 49 55 to 59 60 to 64 65 to 69 Total
54 older
Under 5 6 347 399 246 197 156 163 136 57 16 5 0 1728
5 to 9 0 1 257 326 191 186 232 153 69 34 6 3 1458
10 to 14 0 0 3 214 194 208 274 189 85 47 18 7 1239
15 to 19 0 0 0 4 86 147 133 118 50 26 9 2 575
20 to 24 0 0 0 0 7 529 1081 596 304 145 47 17 2726
25 to 29 0 0 0 0 0 13 583 940 250 103 45 9 1943
30 to 34 0 0 0 0 0 0 21 469 343 121 48 17 1019
35 to 39 0 0 0 0 0 0 0 14 175 112 22 9 332
40 or more 0 0 0 0 0 0 0 0 6 46 72 72 196
Total 6 348 659 790 675 1239 2487 2615 1339 650 272 136 11,216
 Eligible to retire in the next 10 years with full benefits: 6710
 Percent of the NASA Engineering population: 60%
4

5. Looking 10 Years Forward
The year is 2021…
NASA 08xx Series (All Centers)
70 or
ServiceAge Under 20 20 to 24 25 to 29 30 to 34 35 to 39 40 to 44 45 to 49 50 to 54 55 to 59 60 to 64 65 to 69 Total
older
Under 5 4400
4400 new Engineer hires in the next 10 years!
5 to 9
10 to 14 6 347 399 246 197 156 163 136
15 to 19 0 1 257 326 191 186 232 153
20 to 24 0 0 3 214 194 208 274 189
25 to 29 0 0 0 4 86 147 133 118
30 to 34 0 0 0 0 7 529 1081 596
35 to 39
The Senior Tier Retires – ~4400 Engineers
40 or more
Total 6 348 659 790 675 1226 1883 1192 11,216
5

6. What Does this Data Mean to You?
10 years from now, if you are NASA Engineer and are not retired
– GET READY!
ARE YOU READY TO LEAD THIS CHANGE – while managing a challenging and
demanding NASA Project?
You should be, this is an opportunity that affects all NASA Organizations…
This presentation describes how the NASA Safety Center used the
Engineering Lifecycle Model to achieve project success.
6

7. How the NASA Safety Center is Tackling
the Safety & Mission Assurance Challenge
Project Fundamentals and SMA Engineering Technical Excellence
7 http://nsc.nasa.gov/

8. NASA Safety Center
 In 2005, the NASA Safety Center was established in Cleveland, OH
 NASA Headquarters Strategic Guidance -- Build systems to improve
NASA SMA knowledge, information and capability
 Three primary impetus factors:
 CAIB Report
 Demographics
 Need for engineering professionals to be trained in SMA
8

9. NASA Safety Center (NSC)
First and foremost, the NSC is not the NESC or the NSSC
The NSC is comprised of four primary Directorates:
 Technical Excellence
 Knowledge Management Systems
 Audits & Assessments
 Mishap Investigation Support
9

10. SMA and STEP
Safety and Mission Assurance Technical Excellence Program (STEP)
 Agency’s Professional Development Systems for SMA professionals – from
fresh-out to Subject Matter Expert
 Six major SMA Disciplines: System Safety, Quality Engineering, Reliability &
Maintainability, Operational Safety, Software Assurance, and Aviation Safety
 Career-oriented and competency-based
 Appropriate for SMA, Engineering, and Project Managers
 Heavy emphasis on web-based training via SATERN
10

11. STEP’s Overnight Success
 In less than 3 years, STEP has become
recognized Agency-wide as the way SMA
trains
 750 SMA Civil Servants have voluntarily
completed Level 1 (75% of the SMA
population)
 113 SATERN courses developed (650 hours
of engineering-oriented training)
 45,000+ hours of training completed
 Highly successful NASA engineering-oriented
training program
11

12. Success Contributors
How did we accomplish so much so quickly?
 Inspiring leadership
 Dedicated Civil Servant/Contractor team
 Technical Discipline Teams with representatives from each Center SMA
organization (60+ total members)
and, most importantly, a solid process:
The NASA Systems Engineering Project Lifecycle Model (NPR
7120.5)
12

13. Treat it like a NASA Engineering Challenge / Project
The NASA Systems Engineering Project Lifecycle Model (NPR 7120.5)
 This lifecycle model works for Shuttles, Satellites, Airplanes, Automobiles, Ships, etc.
 It also works for Training Program Development – from Formulation to Implementation
13

14. The NASA Engineering Lifecycle Model
Five Primary Phases:
A. Concept & Technical Development
B. Preliminary Design and Technology Completion
C. Final Design & Fabrication
D. System Assembly, Integration, Test and Rollout
E. Operations and Sustainment
14

15. NASA Safety Center’s STEP as an
Engineering Professional Development Model
That’s exactly what we did for the NASA SMA community which accounts
for 10% of the NASA Engineering Workforce…
 We analyzed the problem / challenge
 Then:
 We conceived,
 We designed,
 We fabricated,
 We tested,
 We operated and sustained, and
 (when necessary…) We will closeout.
This model and process helped us evolve NASA’s SMA career-
oriented, professional development system – STEP
15

16. We Started with a Vision
My Vision
Create the NASA University for Safety and ultimately become the Harvard
and MIT of NASA Safety.
Vision Tip
Your Vision should fit on the back of your Business Card (clear and concise)
– Professor Bart Timm, Georgetown University Executive Leadership Program
16

17. We Conceived
In Conceiving, we:
 Looked at successful technical professional development models
 Benchmarked NASA and Industry Safety Training Programs
 Spent time with our customers and stakeholders (NASA SMA and Engineering)
 Engaged NASA Human Resources and the NASA Shared Services Center (NSSC)
 Considered the career life-cycle of our SMA Engineers
 Built six Technical Discipline Teams with representatives from each NASA
Center SMA organization
 Conceived a common framework
 Focused internally on team development and cohesion
 Developed a robust development and implementation schedule with Key
Decision Points, Milestones, Phased Rollout Strategies, and built-in contingency
slack for two of the key events
 Developed a robust formulation to implementation cost analysis
17

18. Engineering Professional Development
Engineers Council for Professional Development (1979)
"Engineering is the profession in which a knowledge of the mathematical and
natural sciences gained by study, experience, and practice is applied with judgment
to develop ways to utilize, economically, the materials and forces of nature for the
benefit of mankind.”
18

19. Concept Result
Safety and Mission Assurance Technical Excellence Program (STEP)
Career-oriented Professional Development
Program (duration: 8 – 10 years)
Six major Discipline Programs
 System Safety
 Software Assurance
 Quality Engineering
 Reliability and Maintainability
 Operational Safety
 Aviation Safety
Components
 Four Qualification Levels
 488 hours of Academics
 1000 hours of OJT
 Comprehensive Test
 Peer-review
Attributes
 Comprehensive and Credible
 Competency-based
 Engineering-oriented
19

20. We Designed
In Designing, we:
 Explored off-the-shelf and advanced technological solutions
 Used Bloom’s Taxonomy (Hierarchy of Learning) for each Discipline
 Developed major competencies, minor competencies, learning and
performance objectives for each minor competency (~250 minor
competencies)
 Documented, documented, documented…
 Peer-reviewed, peer-reviewed, peer-reviewed…
 Assembled NASA HQs independent-review team for the major Milestone
events
 Developed test strategies and included time in the schedule
 Developed minimum success criteria
 Orchestrated Agency SMA organizational momentum strategies
 Briefed every Center SMA Director face-to-face
20

21. Design Results – Competency Wheels
System Safety
6 Disciplines – over 250 Engineering Competencies
21

22. Design Results – Performance Objectives
For each competency, detailed objectives were written to describe exactly what
performance is expected at each level. (Typical performance objectives shown.)

23. Design Results – Curriculums
System Safety Level 2
 Core Training
 Discipline Training
 Readings and Resources
 Domain Training
 OJT
 Enrichment Activities
3/2/2012
23

24. We Developed
In Developing, we:
 Located off-the-shelf, best practice NASA and Industry tutorials and Subject-
matter Experts focused on our courses and learning/performance objectives
 Bought a video integration tool (Mediasite) to capture live courses and
simultaneously integrate video, Powerpoint, and audio into web-based
solutions for SATERN and webcasting
 Reshaped our team to integrate new skill-sets and capabilities
 Rapid-prototyped new course solutions (Build a little / Test a little)
 Collaborated and pooled resources and needs with APPEL and other Center
SMA and Engineering initiatives
 Created licensing agreements with commercial providers to capture SATERN-
ready courses and lectures for unlimited NASA use by Civil Servants and
Contractors
 Created Virtual Instructor-led Courses that are webcasted live training events
24

25. Development Results –
STEP Course Delivery Methodology
Levels 2-4 Courses offerings are:
 Web-based e-Learning (WBT) via
SATERN
 Existing NASA instructor-led classroom
training (ILT)
 Videotaped ILT lectures using Sonic
Foundy’s Mediasite technology
 Virtual ILT courses using Webex and
telecon
 External WBT and ILT courses
(Industry/Commercial Providers)
25

26. We Assembled and Tested
In Assembly and Test, we:
 Performed subsystem tests on the individual courses and learner performance
tests
 Simulated the operational in environment through internal (NSC) and external
(GRC & MSFC SMA) full Beta-test runs of the Level 1 & Level 2 Curriculums in
SATERN
 Refined the courses and implementation based on Beta-test results and
feedback
Implementation/Operations Preparation:
 We planned and simulated a full-scale rollout that included an Agency-wide
SMA webcast that was attended by 1000 participants
 The Program Manager personally visited every NASA Center and HQs in the
four months prior to Program rollout and commissioning and met with each
Center SMA Director and their senior leadership team
26

27. We Conducted Operations
In Operations from 2009-2011:
 Provided over 45,000 hours of web-based and instructor-led NASA-
oriented training
 Level 1 (Graduated/Enrolled): 2143 / 2246 (95% of the Active Learners)
 Level 2
(Graduated/Enrolled): 29 /
331 (9%)
 Level 3 Graduated (CS): 1
 Civil Servants Active in Levels
2 – 4 (2011): 193
 Civil Servants Hours Levels
2 – 4 (2011): 18,708.5
(includes OJT)
27

28. STEP - A NASA SMA Engineering Transformation
Impact of Web-Based Training
SMA Instructor-Led & Web-Based Learners (July 2006 – December 2010)
Pre-STEP STEP
- 4400% increase in web-based training in the first operational Quarter of STEP Level 1!
- 550% increase in NASA’s overall Safety training usage

29. In Summary
The quest for NASA Technical Excellence is never ending and
the Organization’s Professional development system must be
continuously assessed from both the capability and strategic
application perspectives…
 We strategically apply the STEP Program to the SMA Community
 We continuously measure performance
 We listen to the voice of the customer (Learners) and stakeholders
(SMA Leaders)
29

30. Parting Thoughts to Ponder
As a NASA Project and Engineering Leader…
 Are you using APPEL and other Engineering Professional Development
capabilities from a strategic perspective (do you have learning and
development goals for each team member associated with team goals)
or are team members choosing their own destiny (which may or may
not be where the team needs them to be)?
 Is your organization truly a driven and optimized learning organization?
 Are your engineers using techniques from the 1980’s or are they using
today’s industry and government best practices?
30

31. I’m interested in your thoughts and
some discussion…
31 http://nsc.nasa.gov/