John West from the DoD HPC Modernization Program presents an Update on What’s Missing from HPC? You can watch the presentation here: http://insidehpc.com/2013/03/26/update-john-west-on-whats-missing-from-hpc

1. Update: What’s Missing from
HPC?
John West
DoD HPC Modernization Program

2. If you’re in this room, you
probably think HPC is a great idea.
HPCC Newport 2013
Page-2

3. Peak Computational Capability by Country
And as we heard last year 1993
countries all over the
world are racing to close
the leadership gap in
capabilities once held by
just a small number of
nations.
2012
Data : top500.org HPCC Newport 2013
Page-3

4. Technical Computing
Supercomputing Few users
High Performance Few users
Computing
“Missing Middle”
Individual Many users
Computing
HPCC Newport 2013
Page-4

5. So…where are they?
HPCC Newport 2013
Page-5

6. HPCC Newport 2013
Page-6

7. HPCC Newport 2013
Page-7

8. HPCC Newport 2013
Page-8

9. The HPC and
supercomputing
users we have
today didn’t have a
choice.
HPCC Newport 2013
Page-9

10. The choice was
obvious for
individual computer
users.
And the middle,
stayed at the
bottom.
HPCC Newport 2013
Page-10

11. The journey from the bottom.
HPCC Newport 2013
Page-11

12. HPC User
• Incomplete toolchain
• Little expertise and no social support
• Primitive interfaces
• Complex management
• Expensive hardware and software
Individual Computing
User HPCC Newport 2013
Page-12

13. Why do they stay at the bottom?
They already have something
that works, and it’s too hard to
just “take ‘er for a spin.”
HPCC Newport 2013
Page-13

14. How do we increase the reach of HPC?
Hardware has gotten cheaper and better
There are system management options that
reduce deployment complexity
 Interfaces are primitive (some work needed)
 Incomplete toolchain (hand to hand combat)
Little expertise and no social support
“I don’t know how, and there’s no one around here
I can ask!”
HPCC Newport 2013
Page-14

15. Makers and Takers
 HPC Consumer
– Use high performance computers
– Run applications
– Understand computing principles
 HPC Provider
– Run and design high performance computers
– Write and extend applications
– Master computing principles
In practice this is a continuous spectrum, and
workers may move in either direction during their
career.
HPCC Newport 2013
Page-15

16. An interagency (NITRD) position
 The NITRD High End
Computing Interagency
Working Group (HEC-IWG)
position on education and
workforce development
(Mar 2013)
 Articulates foundational
principles
 Starting place for coordinated
agency programs that will
build a workforce
 NITRD is not a funding
agency
The Networking and Information Technology Research and Development
Program, www.nitrd.gov
HPCC Newport 2013
Page-16

17. NITRD Position Overview
 Affirms the importance of HPC/HEC in national
security and competitiveness terms
 Reviews DOE/NNSA-funded survey on
characteristics of the HEC workforce
– Statistics remain a problem for this segment of the workforce
 Articulates foundational principles that must be
addressed for success
HPCC Newport 2013
Page-17

18. DoE HPC Provider Study
IDC HPC User Forum: Special Study (July 2010). A Study of the Talent
and Skill Set Issues Impacting HPC Data Centers.
 Staffing is hard
– 93% of HPC centers surveyed said that hiring qualified staff is
“somewhat hard” or “very hard” with the majority reporting that it is “very
hard” to find qualified staff.
 Where do staff come from?
– STEM grads
– Other HPC centers
– HPC vendors
 What skills are needed on the provider side?
– Combined understanding of a scientific discipline and computational
science and/or computer science; parallel programming and code
optimization, especially for scaling to large processor/core counts;
algorithm development; HEC system administration; and understanding
of parallel file systems.
HPCC Newport 2013
Page-18

19. The NITRD Principles
 An effective program
– Increases the impact of HPC/HEC
– Must address the entire spectrum consumer  provider
HPCC Newport 2013
Page-19

20. The NITRD Principles
 An effective program
– Increases the impact of HPC/HEC
– Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
just as important as increasing STEM grads
– Many of us came to HPC after practicing in a discipline that uses it
– Steal whenever possible (executive MBAs, certificates, …)
HPCC Newport 2013
Page-20

21. The NITRD Principles
 An effective program
– Increases the impact of HPC/HEC
– Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
just as important as increasing STEM grads
 If we want to teach it we have to define it
– Enumerate the skill vectors that span our space (steal when possible):
admins, architects, developers, …
– Then work with traditional and non-traditional education partners on
curricula
HPCC Newport 2013
Page-21

22. The NITRD Principles
 An effective program
– Increases the impact of HPC/HEC
– Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
just as important as increasing STEM grads
 If we want to teach it we have to define it
 …and reinforce it
– (Continue to) fund research that gives academic community experience
with real world (ish) problems
– Internships, fellowships, awards, etc.
– Establishing and illuminating HPC career paths will help with recruitment
and retention (certifications? Maybe eventually…)
HPCC Newport 2013
Page-22

23. Next Steps for NITRD HEC Members
 Define a set of career paths and skillsets
 Map the union of current efforts, identify gaps
 With educators, describe and develop curricula that
will produce new Providers and Consumers
 Pilot new, more flexible methods of education and
workforce development that enable in-career
transitions
 Continue to fund relevant academic research
problems, internships, graduate and post-doctoral
fellowships, and partnerships with industry and
academia.
...and share, share, share
HPCC Newport 2013
Page-23

24. Read it at goo.gl/e03fU
Comment at john.west@hpc.mil
HPCC Newport 2013
Page-24