IBM recently completed a benchmarking study of several key modules of the Accelrys Enterprise Platform (AEP) 9.0, using IBM’s iDataPlex and General Parallel File System (GPFS). The results show that the performance of IO intensive workloads, such as Next Generation Sequencing (NGS), can be improved significantly by using GPFS. NGS workloads can also benefit from better load balancing implemented on AEP 9.0. Best practices for scalable IT solutions will also be discussed.

1. Performance Analysis of AccelrysPerformance Analysis of Accelrys
Enterprise Platform 9.0 on IBMEnterprise Platform 9.0 on IBM’’ss
Scalable IT solutionScalable IT solution
Kathy Tzeng, PhD
Senior Technical Staff Member
IBM
tzy@us.ibm.com

2. © 2013 IBM Corporation
Content
• Performance of AEP 9.0
– NGS collection: Bowtie, Bowtie2, BWA
• Effect of File System on Performance
– NGS Collection: Bowtie2
– Chemistry: Energy Minimization
• Best Practice
– Reference Architecture

3. © 2013 IBM Corporation
Performance of AEP 9.0

4. © 2013 IBM Corporation
System Configuration

5. © 2013 IBM Corporation
Test Case Description
• Mapping Module:
– Bowtie, Bowtie2 and BWA Mapper in the NGS Collection
• Experiment:
– SRX000600, Illumina sequencing of Human HapMap individual NA18507
genomic paired-end library
– 213 runs, 1.6 billion spots, 117.4 G bases
– Input: ~158GB compressed (~600GB uncompressed) FASTQ files
– Output: 180 GB BAM files (~600 GB tmp files)
• Reference:
– Full Human Genome (3GB) build 37.3

6. © 2013 IBM Corporation
Bowtie Mapping Protocol

7. © 2013 IBM Corporation
Performance of Bowtie Protocol
303
160
92
65
213
135 125
187
97
53
36
0
50
100
150
200
250
300
350
8 16 32 64
Number of Nodes
ElapsedTime(minutes)
Bowtie: 1 process/node, 32 threads/process
Bowtie: 4 process/node, 8 threads/process
Bowtie: 4 process/node, 8 threads/process w/ Splitter
on 2.6 GHz iDataPlex with GPFS

8. © 2013 IBM Corporation
Scalability Analysis of Bowtie Protocol
Bowtie benchmark comparative performance
187
97
53
36
1.00
1.93
3.53
5.19
0
20
40
60
80
100
120
140
160
180
200
8.0 16.0 32.0 64.0
Number of Nodes
ElapsedTime(seconds)
1
10
RelativeSpeedup
Bow tie Runtime
Bow tie Speedup
Linear Scale
on 2.6 GHz iDataPlex with GPFS

9. © 2013 IBM Corporation
Bowtie 2 Mapping Protocol

10. © 2013 IBM Corporation
Performance of Bowtie2 protocol
385
189
119
82
301
196 189
254
129
69
48
0
50
100
150
200
250
300
350
400
450
8 16 32 64
Number of Nodes
ElapsedTime(minutes)
Bowtie2: 1 process/node, 32 threads/process
Bowtie2: 4 process/node, 8 threads/process
Bowtie2: 4 process/node, 8 threads/process w/ Splitter
on 2.6 GHz iDataPlex with GPFS

11. © 2013 IBM Corporation
Bowtie2 benchmark comparative performance
254
129
69
48
1.00
1.97
3.68
5.29
0
50
100
150
200
250
300
8.0 16.0 32.0 64.0
Number of Nodes
ElapsedTime(seconds)
1
10
RelativeSpeedup
Bow tie2 Runtime
Bow tie2 Speedup
Linear Scale
Scalability Analysis of Bowtie2 Protocol
on 2.6 GHz iDataPlex with GPFS

12. © 2013 IBM Corporation
BWA Mapping Protocol

13. © 2013 IBM Corporation
Performance of BWA protocol
305
162
93
74
179
100
75 75
0
50
100
150
200
250
300
350
8 16 32 64
Number of Nodes
ElapsedTime(minutes)
BWA: 4 process/node, 8 threads/process w/ Splitter
BWA: 8 process/node, e threads/process w/ Splitter4
on 2.6 GHz iDataPlex with GPFS

14. © 2013 IBM Corporation
Scalability Analysis of BWA Protocol
BWA benchmark comparative performance
179
100
75 75
1.00
1.79
2.39 2.39
0
20
40
60
80
100
120
140
160
180
200
8.0 16.0 32.0 64.0
Number of Nodes
ElapsedTime(seconds)
1
10
RelativeSpeedup
BWA Runtime
BWA Speedup
Linear Scale
on 2.6 GHz iDataPlex with GPFS

15. © 2013 IBM Corporation
Summary of performance of NGS collection
Elapsed time for mapping 30x coverage of human genome
on 2.6 GHz iDataPlex with GPFS

16. © 2013 IBM Corporation
Effect of File System on Performance

17. © 2013 IBM Corporation
File System Description
4 x DCS3700 (240 x 2TB, SAS, 7.2K RPM)/vol/xcae1
DDN SFA12k (280 x 3TB, SAS, 7.2K RPM)/gpfs
Storage SystemFile System

18. © 2013 IBM Corporation
Energy Minimization Test Cases
Energy Minimization Protocol with SD Files
Energy Minimization Protocol without SD Files

19. © 2013 IBM Corporation
Energy Minimization Benchmarks
228227No SD file
629356Write SD file
/vol/xcae1/gpfsTest Case
* Benchmark was run by using 32 processes on single node
on 2.6 GHz iDataPlex with GPFS and NFS
Elapsed Time in seconds, lower is better

20. © 2013 IBM Corporation
Bowtie2 Benchmarks
437119Bowtie2
/vol/xcae1/gpfsTest Case
on 2.6 GHz iDataPlex with GPFS and NFS
Elapsed Time in Minutes, lower is better
* Benchmark was run by using 32 nodes

21. © 2013 IBM Corporation
File Cache Library Improves I/O Performance
Without cache library With cache libraryWithout cache libraryWithout cache library With cache libraryWithout cache library
Elapsed Time = 1730 min ElapsedTime = 107 min

22. © 2013 IBM Corporation
Best Practice

23. © 2013 IBM Corporation
IBM Reference Architecture for AEP

24. © 2013 IBM Corporation
• Refined / matured for past 13 years
• Over 100,000 GPFS licenses worldwide
• In academic, commercial and government
• Proven scalability for thousands of nodes
• Proven capability for greater than 10 PB in single file system
• File system performance > 1,500 GB/sec
• Amount of archival storage > 500 PB
• http://www-03.ibm.com/systems/software/gpfs/
IBM General Parallel File System

25. © 2013 IBM Corporation
Q & A