SERC, IISc CRAY PetaFLOPS System
•Download as PPTX, PDF•
2 likes•3,210 views
The Supercomputer Education and Research Center (SERC) at the Indian Institute of Science, Bangalore (IISc) procured the fastest supercomuter in India and 50-60th fastest in the world. These are some images in setting up the machine and its supporting datacenter requirement More details here: http://adarshpatil.in/timewarp/blog/iisc-serc-cray-supercomputer-sahasrat.html
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to SERC, IISc CRAY PetaFLOPS System
Similar to SERC, IISc CRAY PetaFLOPS System (20)
Recently uploaded
Recently uploaded (20)
SERC, IISc CRAY PetaFLOPS System
- 1. May 11, 2015 SERC Petascale System
- 2. SERC Providing computing facility to Institute users from 1970. State-of-the-art facility for parallel computing with high performance storage along with a range of software application domains. Balance between capacity and capability computing to meet diverse requirements of users.
- 3. SERC Petascale System Cray-XC40, an MPP class machine Petascale compute capability with ◦ CPU Clusters with 33024 Intel-Haswell processor cores achieving more than 900 TFLOPS ◦ 44 node GPU clusters with Intel Ivybridge processor and Tesla K40 GPU, delivering 52 TFLOPS ◦ 48 node Intel Xeon Phi Cluster (5120D) giving 28 TFLOPS
- 4. Compute Blade : Each having 4 nodes or 96 cores Chassis : 16 blades; 64 nodes; No cables Group: 6 Chassis and 384 nodes Electrical network cables System 4 Groups Active Optical cables SYSTEM BUILDING BLOCKS
- 5. Aries NOC Connects nodes on a blade 128Gbps Rank 1 Network Backplane connection through blade Connects nodes on a chassis 157.6Gbps Rank 2 Network Passive Electrical cables Connect multiple chassis All-to-all chassis connection 157.6Gbps Rank 3 Network Optical cable connection to multiple cabinets All-to-all cabinet connection on a Dragonfly topology 43Gbps SYSTEM INTERCONNECT
- 7. Parallel File System 2 Peta Byte of Direct Attached parallel storage with RAID6 from DDN Cray’s Parallel Lustre File System 28 GBps Read and 32 GBps Write performance
- 8. Installation & Commissioning of the Petascale System
- 9. Petascale System: Status Operational from Jan. 15, 2015 System Acceptance by Feb. 19, 2015 60+ users A few show case capability applications
- 10. Unsteady Aerodynamic Simulation Dynamic Ground Effect: Unsteady simulation of entire landing sequence of a high lift wing Problem size: 36 Million volume grid & simulation of 11 secs of the landing sequence System : 8 days on XC40 using 10000 Xeon cores (for a time step of .001 sec)
- 11. Observation: A critical number of supernovae required to maintain an over-pressured bubble for 10s of Myr, for a given density and spatial separation between supernovae. These superbubbles should break out of galaxies and suppress global star formation. PLUTO hydrodynamics code to simulate supernovae using 5123 and 10243 simulations. Application scales upto 12,000 cores. Simulating Overlapping Supernovae
- 12. Thank You!
Editor's Notes
- SERC has been providing the state-of-art computing facility to the Institute community. The facility enables users to access specialized parallel computing systems with high performance storage, using wide-range of application software. SERC strives to meet the varied computing demands ranging from capability to capacity needs. The facility is operated round-the-clock, 365 days and is supported by uninterrupted power and air-conditioning.
- In its endeavor to bring the best-in-class computing, SERC has recently procured the Cray-XC40 system consisting of pure CPU or accelerator based computing. This system has three major computing clusters, namely Intel-Haswell processor based CPU cluster with sustained HPL performance of around 950 TFLOPS from 1376 nodes. Nvidia K40 GPU card based accelerator nodes with a sustained performance of 52TFLOPS from 44 nodes. Intel Xeon-Phi co-processor based accelerator nodes with a sustained performance of 28TFLOPS from 48 nodes. These put-together deliver a performance of more than 1000 TeraFLOPS. The cluster components are seamlessly tied with the Aries interconnection fabric on a dragonfly topology, delivering the fastest bisection bandwidth to applications.
- The XC40 system is built using scalable system building blocks. The basic unit of the system is called a compute blade that is composed of four compute nodes. Sixteen compute blades integrate on a back plane to compose a system Chassis. Three chassis units form a cabinet. SERC’s petascale system is built using eight cabinets.
- The system building blocks are tied up using the 48-tiled Aries network-on-chip interconnect. One Aries NOC connects the four compute nodes on a blade. These blades are the connected using the same Aries NOC onto a backplane forming the Rank-I communication layer within a chassis. Two cabinets chassis’ are inter-connected on an all-to-all basis using passive electrical cables on the Rank-II communication layer. Two cabinets form a group and multiple groups are interconnected on an all-to-all mode using optical cables. The system uses dragonfly inter-connection topology.
- The system software is built on Cray’s customized Linux operating system The XC40 systems hosts a whole range of parallel program development tools and architecture specific compilers and parallel scientific and mathematical libraries.
- The machine also hosts the largest directly attached high-speed parallel storage of 2PB.
- Lets look at some images in the journey of the installation and commissioning of CRAY at SERC between the 13 and 15th december
- The machine is operational from January 15 of this year. It has got more that 60 users currently; We will demonstrate a few show case applications that exploit the capability of this large system;
- This user group simulates the entire landing sequence of a high life wing, which requires simulation of complex physics for large grids. They have simulated 36 million volume grid for more than 11 simulation seconds of landing, at a granularity of 1 millisecond, using 10000 cores. The three distinct phases -- gliding and flaring where the wind incidence increases & post-touch down phase where the wind incidence is lost while on ground roll. The movie depicts trailing vortices off the wing-tip and part-span flap end. One can notice the vortex intensifying during the flaring operation.
- In this work the user group has simulated the overlap of supernovae that forms a hot, over-pressured Bubble using the public domain PLUTO hydrodynamics code using large machine configuration, Currently upto 12,000 cores, which is roughly one-third of the system.