1. Tuning Parallel Applications to
Accelerate Scientific Discoveries
Robert Henschel
rhensche@indiana.edu
October 2009

2. Contents
• PTI / High Performance Applications
• Performance of Scientific Codes
• IU and TeraGrid Compute Resources
• Optimizing for IU's HPC Systems
• Using TeraGrid HPC Systems
• HPA is Here to Help
Robert Henschel

3. What this talk will be about
• Making you aware of compute resources that you
can use for your work, to make you more
productive.
• Introducing the High Performance Applications
group and how we can help get research done
faster.
• Give you examples of what we have done for
researchers to make them more competitive in their
field.
Robert Henschel

4. PTI and High Performance Applications
• Pervasive Technology Institute
– Develop and deliver innovative information technology
to advance research, education, industry and society.
– School of Informatics
– School of Law
– University Information Technology Services
• High Performance Applications
– Part of the Digital Science Center of PTI
– Part of the Research Technologies of UITS
– Seven people that help IU researchers make efficient
use of IU and TeraGrid compute resources
Robert Henschel

5. Performance of Scientific Codes
• Supercomputing, or High Performance Computing (HPC),
is not just for computer geeks!
• Performance for computer scientists
– Amdahls law and scalability
– Efficient usage of functional units of processors
– Optimally using memory bandwidth
– Trying to avoid I/O as much as possible
• Performance for researchers
– When do I get the answer to my problem?
– When does my job run and when is it done?
Robert Henschel

6. IU and TeraGrid Compute Resources
• Two HPC systems at IU
– BigRed 30 TFLOPS (3000 cores)
– Quarry 7 TFLOPS (1000 cores)
• Several special purpose systems
– Small Cell B.E. Cluster
– MDGRAPE-2 machine
• Several storage resources
– IU Data Capacitor
– GPFS, RFS, HPSS
• Policy of open access to compute resources
Robert Henschel

7. IU and TeraGrid Compute Resources cont'd
Robert Henschel

8. IU and TeraGrid Compute Resources cont'd
• TeraGrid
– NSF funded HPC systems and support infrastructure
– 11 resource providers
– More than 1,500 TFLOPS (150,000 cores)
• Central allocation and support structure
Robert Henschel

9. Optimizing for IU's HPC Systems
• Help researchers access the central systems and
determine what system to use
• Install and optimize applications
• Provide guidance on compiler and library optimization
• Help with job submission, especially running many
thousands of jobs
Robert Henschel

10. Using TeraGrid HPC Systems
• Low barrier of entry
• Identify if a problem and workflow will work on the
TeraGrid
• Get a startup allocation
• Use it and identify if it is worth pursuing this further
• Submit a full allocation request
Robert Henschel

11. Contents – HPA is Here to Help
• HPA is Here to Help
– What We Do
• Recent Examples
– Integrating HPC Systems into an Electron
Microscope Workflow
– Migrating Research in Gas Giant Planets from IU
to TeraGrid HPC Systems
– Developing Computational Models to Predict
Drug-Drug Interactions
Robert Henschel

12. What We Do
• Consulting about HPC system usage
– From start to finish
– Optimize source code for architectures
• Help with TeraGrid allocation proposals
• Adapting and creating workflows for new environments
• Consulting for grant proposals
Robert Henschel

13. HPC Systems and an Electron Microscope
General Case
– Users have an instrument that produces a lot of data
on a daily basis
– This data needs to be stored and analyzed
Electron Microscope in Simon Hall (IU Bloomington)
– Microscope stores data on a Windows workstation
– Researcher does quality checks on local workstation
– IU Data Capacitor links workstations, IU HPC systems
and the IU long term archive together
Robert Henschel

14. Gas Giant Planets on the TeraGrid
General Case
– Users have a set workflow for analyzing data
– Locally available compute resources are not big
enough to keep up with demand
Understanding Gas Giant Planets
– IDL is used to visualize simulation data
• Commercial software, IU Astronomy has a license
– Simulation application needs to run on a large shared
memory system
– TeraGrid and IU Data Capacitor tie this workflow
together
Robert Henschel

15. Predicting Drug-Drug Interactions
General Case
– Researchers implement proof of concept research
algorithms
– Scaling from proof of concept to production science is
difficult
– The ability to add HPC expertise to grant proposals will
make the proposal more competitive
Computational Models to Predict Drug-Drug Interactions
– Drug exposure model developed in R
– Scaling to real world data sets not possible without
using HPC systems
– Porting to C and running on UITS hardware
Robert Henschel

16. What this talk was about
• Made you aware of compute resources that you can
use for your work, to make you more productive.
• Introduced the High Performance Applications
group and how we can help get research done
faster.
• Gave you examples of what we have done for
researchers to make them more competitive in their
field.
Robert Henschel

17. Acknowledgments
This material is based upon work supported by the National Science
Foundation under Grant Numbers 0116050 and 0521433. Any
opinions, findings and conclusions or recommendations expressed in
this material are those of the author and do not necessarily reflect the
views of the National Science Foundation (NSF).
This work was support in part by the Indiana Metabolomics and
Cytomics Initiative (METACyt). METACyt is supported in part by Lilly
Endowment, Inc.
This work was support in part by the Indiana Genomics Initiative. The
Indiana Genomics Initiative of Indiana University is supported in part by
Lilly Endowment, Inc.
This work was supported in part by Shared University Research grants
from IBM, Inc. to Indiana University.
Robert Henschel