Unleash oracle 12c performance with cisco ucs

1. Unleash Oracle 12c Performance with
Cisco UCS
Download this slide
http://ouo.io/cbI00
Aprils 2014

2. Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle Database 12C – Unleashing Performance
• Goal: Define and Develop blueprint for
Multitenant Database architecture
• Best practices for “Pluggable Databases”
• Flex ASM - Allows to separate ASM from
databases
• Multitenant design running OLTP, DSS and
in memory databases
• Traditional database and containers running
together
System Resources
CPU
IO
DSS
DSS

3. Cisco Confidential 3© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cisco UCS: Many Server Form Factors, One System
Industry-Leading Compute Without Compromise
Current UCS Compute Portfolio
Performance Optimized for Bare Metal, Virtualized, and Cloud Applications
UCS C220 M3
Versatile, General Purpose Enterprise
Infrastructure, and Application Server
Enterprise Performance
UCS C240 M3
Ideal Platform for Big Data, ERP,
and Database Applications
UCS B200 M3
Optimal Choice for VDI, Private
Cloud, or Dense Virtualization/
Consolidation Workloads
Intensive/Mission Critical
UCS B420 M3
Enterprise Class, 4-Socket Blade for
Large, Memory-Intensive Bare Metal
and Virtualized Applications
UCS C420 M3
Enterprise Class, 4-Socket Server
for Large, Memory-Intensive Bare
Metal, and Virtualized Applications
UCS C24 M3
Entry, Expandable Rack Server for
Storage Intensive Workloads
UCS C22 M3
Entry Rack Server for Distributed and
Web Infrastructure Applications
UCS B22 M3
Entry Blade Server for IT
Infrastructure and Web Applications
Scale Out
RackBlade
UCS C260 M2
Mission-Critical, 2-Socket Extended
Memory Server for Large, Memory-
Intensive Applications
UCS C460 M2
Mission-Critical, 4-Socket
Server for Large, CPU-Intensive
Applications
UCS B440 M2
Mission-Critical, 4-Socket Blade for
Large, CPU-Intensive Bare Metal
and Virtualized ApplicationsUCS B230 M2
Density-optimized CPU and
Memory-Intensive 2-Socket Blade for
Bare Metal and Virtualized
Applications
Available

4. Cisco Confidential 4© 2013-2014 Cisco and/or its affiliates. All rights reserved.
 UCS
 B420M3 4 Socket Servers @ 256 GB RAM
 4 x 8 Gbps vHBA and 2 x 10Gbps vNICS on each blade
 Storage
 EMC VNX 8000.
 256 SAS, 50 SAS flash drives and 600GB of flash Cache.
 32 x 8Gb FC Connections through N5K.
 OS and Databases
 Linux uek2 r2 kernel with EMC PowerPath.
 Oracle CRS and RDBMS 12.1.0.1.2 along with ACFS
 OLTP database – 4TB
 DSS database – 6 TB
 CPU and CRS – 1 TB
 Workloads
 Oracle OAST workload for performance and stress tests.
Oracle 12c Certification Architecture

5. Cisco Confidential 5© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Hardware Calibration – Orion IOPS and Bandwidth tests
• 8 node Oracle RAC
cluster
• 2 chassis, 8 x B420
Blade servers
• 750K IOPs at very low
latency (< 2 ms)
• 25 GBytes/Sec.
bandwidth (200 Gbps)
• Linear Scalability
IOPs and
Bandwidth as
cluster nodes
added
102,863
205,591
319,238
409,468
489,248
561,253
633,049
751,502
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
1 2 3 4 5 6 7 8
IOPS
Nodes
Read IOPS
IOPs
3,078
7,013
9,703
11,273
14,801
16,731
20,184
24,892
0
2,500
5,000
7,500
10,000
12,500
15,000
17,500
20,000
22,500
25,000
1 2 3 4 5 6 7 8
BandwidthMB/Sec
Nodes
Read MBPS
Bandwidth

6. Cisco Confidential 6© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle Orion Tests – Running OLTP and DSS together
OLTP workload
• Random 8K Reads
• 70% Reads, 30% Writes
• 330K IOPs at very low latency
(< 2 ms)
• Storage Pool with 50 SSDs
DSS workload
• Large Reads 1 MB size
• 90% Reads, 10% writes
• 17GB/Sec Bandwidth
• Storage Pool with 25
SSDs,160 SAS drives
52,138
89,209
107,392
179,243
252,615
295,728
309,770
327,449
0
50000
100000
150000
200000
250000
300000
350000
1 2 3 4 5 6 7 8
ThroughputinIOPS
Number of Nodes
OLTP Luns IOPS - 30% writes @8k IO Size
1 2 3 4 5 6 7 8
3,284
7,013
9,703
11,273
14,801
15,730 16,184
17,738
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
1 2 3 4 5 6 7 8
BandwidthinMBPS
Number of Nodes
DSS Luns Throughput - 10% writes @1MB IO Size

7. Cisco Confidential 7© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle 12c 8-Node UCS cluster - Logical Architecture
ASM Flex Cluster
OLTP Disk Group CPUCRS Disk Group REDO Disk Group OCRVOTE Disk GroupDSS Disk Group
APXAPXASM+
APX
ASM+
APX
ASM+
APX
APXAPX APX
Scan
Listeners
8 Node UCS cluster
• UEK2 R2 Linux kernel
• 12.1.0.1 CRS on 8 hub nodes
with Flex ASM

8. Cisco Confidential 8© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Multiple Containers vs Single Container with PDBs
TPM – 1,162,648
Throughput = 154,864 IOPS
Bandwidth = 10,356 MB/sec
Interconnect = 2,426 MB/sec
Interconnect
Non-CDB
database
Interconnect
Non-CDB
database
TPM – 1,241,552
Throughput = 132,570 IOPS
Bandwidth = 10,270 MB/sec
Interconnect = 2,377 MB/sec
CPU
DSS
DSS (6000 GB)
OLTP (4000GB)
CPU Stress (500 GB)
DSS PDB (6000 GB)
OLTP PDB (4000GB)
CPU Stress PDB (500 GB))

9. Cisco Confidential 9© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Stress Tests
Oratop snapshot

10. Cisco Confidential 10© 2013-2014 Cisco and/or its affiliates. All rights reserved.
EM Cloud 12c Performance Data

11. Cisco Confidential 11© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Destructive Tests

12. Cisco Confidential 12© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle RAC on Cisco UCS – Improved Availability
Failure Description Response
1 or 2 or 3 or
4
Single Link Failure
(Public Interface)
Represents Port Failure—
nodes Should Continue to
Work
1 and 2 or
3 and 4
All Links Failure
(Public Interface)
Nodes Should Continue to
Work via Failover
5 or 6 or 7 or
8
Single Link
(Private
Interconnect)
Represents Port Failure —
Nodes Should Continue
to Work
5 and 6 or 7
and 8
All Links (Private
Interconnect)
Nodes Should Continue to
Work via Failover
9
Storage Path
Failure
1 IO Path Lost—No Effect,
All IO Paths— Nodes
Should Reboot
Common Failures
tested at high
stress levels

13. Cisco Confidential 13© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Flex ASM Failover
ASM Flex Cluster
Disk Group 1 Disk Group 2
Node1 Node2 Node3 Node4
ASM Flex Cluster
Disk Group 1 Disk Group 2
Node1 Node2 Node3 Node4
Node 1 ASM killed
ASM Instance migrates
12c DB on Node1 continues to operate

14. Cisco Confidential 14© 2013-2014 Cisco and/or its affiliates. All rights reserved.
 All Oracle products are certified with Cisco UCS
 Oracle RAC extensively tested on Unified Fabric Oracle 10g R2/11g R2/12c
 Best Practices determined via Cisco Validated Designs and studies
 Focus on application benchmarks, sizing data entered into sizing tools
 Integrated Solutions (Vblock, Flexpod) fully documented with Oracle
 Customer successes documented – TCO, Scaling, Performance
 Majority of activity performed in consultation with Oracle
Cisco Validated Solutions
for Oracle
Sizing & Best Practice
Guides, Scaling Studies,
Product Certifications
Customer Case Studies and
TCO Analysis
Oracle Focus Areas
Oracle Application, Database
and Industry Standard
Benchmarks

15. Thank you.

16. Cisco Confidential 16© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Hardware Destructive Tests
Test Status
1 Multiple Network
Connection Failures
Ramp up to full load. Disconnect 2xpublic
(chassis1) and 2xprivate (chassis2 ).
Chassis 2 servers
reboot and join cluster.
2 Network failures
between FI and
Corporate Network
Ramp the system to full load. Disconnect
connection from each outbound switch one
after the other.
No disruption to Oracle
clients.
3 Fabric Failover
Tests
Run the system on full load. Reboot FabricA,
followed by FabricB.
Fabric failovers should
not cause disruption (
FC output is reduced )
4 Path Failure Tests Run the system on full load. Disconnect the FC
connections. Reconnect them back during
restart.
Servers rebooted and
joined back the cluster.
5 Swap the blades Under full load, swap out one blade from each
chassis
After rediscovery, the
blades joined the
cluster.

17. Cisco Confidential 17© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Hardware Destructive Tests
Test Status
6 ACFS Test1 Kill the volume driver process vdbg under
full load, on couple of nodes, that handles
extents locks and other volume
management functions.
DB instances crash as the
underlying acfs files
system is not available.
7 ACFS Test2 Ramp up the system load and enlarge the
ACFS file system
No disruption to Oracle
clients.
8 ACFS Test3 Kill the ASM CKPT process under full load Process re-spawned and
workload continues.
9 Flex ASM Send sigkill to the asm pmon process
when running under full load.
ASM process migrates to
another node and there is
no interruption to Oracle
Work load

18. Cisco Confidential 18© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Performance Data
Time Recv Send usr % sys % idl % wai %
7:30:11 242M 224M 43 16 29 10
7:30:11 145M 217M 48 16 17 16
7:30:11 244M 177M 42 16 21 18
7:30:11 252M 154M 44 16 26 11
7:30:11 301M 123M 44 16 20 17
7:30:11 259M 343M 46 17 23 11
7:30:11 259M 342M 45 17 25 10
1702M 1580M
AWR Data
Interconnect from EM Cloud control
Interconnect from dstat

19. Cisco Confidential 19© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Hardware Calibration – Orion IOPS and Bandwidth tests
• 8 node Oracle RAC
cluster
• 2 chassis, 8 x B420
Blade servers
• 750K IOPs at very low
latency (< 2 ms)
• 25 GBytes/Sec.
bandwidth (200 Gbps)
3,078
7,013
9,703
11,273
14,801
15,731
18,184
23,892
102,863
205,591
319,238
409,468
489,248
561,253
633,049
731,502
0
5,000
10,000
15,000
20,000
25,000
30,000
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
1 2 3 4 5 6 7 8
MBPS
IOPS
Nodes
Bandwidth
Throughput

20. Cisco Confidential 20© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle Orion Tests at expected workloads
OLTP workload
• Random 8K Reads
• 70% Reads, 30% Writes
• 330K IOPs at very low latency (< 2
ms)
• Storage Pool with 50 SSDs
DSS workload
• Large Reads 1 MB size
• 90% Reads, 10% writes
• 17GB/Sec Bandwidth
• Storage Pool with 25
SSDs,160 SAS drives

21. Cisco Confidential 21© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Oracle 12c 8-Node UCS cluster - Logical Architecture
ASM Flex Cluster
OLTP Disk Group CPUCRS Disk Group REDO Disk Group OCRVOTE Disk GroupDSS Disk Group
ASM+
APX
ASM+
APX
ASM+
APX
APXAPX APX APX APX
Scan Listeners
8 Node UCS cluster
• UEK2 R2 Linux kernel
• 12.1.0.1 CRS on 8 hub nodes
with Flex ASM

22. Cisco Confidential 22© 2013-2014 Cisco and/or its affiliates. All rights reserved.
UCS improved availability Oracle RAC
 Normal Operating conditions
• Private interconnect traffic is
localized
• Extremely Low LatencyFailFail

23. Cisco Confidential 23© 2013-2014 Cisco and/or its affiliates. All rights reserved.
UCS improved availability Oracle RAC
Fail Fail

24. Cisco Confidential 24© 2013-2014 Cisco and/or its affiliates. All rights reserved.
CPU
IO

25. Cisco Confidential 25© 2013-2014 Cisco and/or its affiliates. All rights reserved.
CDB vs PDB Performance
TPM – 1,162,648
Throughput = 224,864 IOPS
Bandwidth = 10,356 MB/sec
Interconnect = 2,426 MB/sec
OLTP
CDB
CPU
CDB
CRS
non-CDB
DSS
CDB
TPM – 1,241,552
Throughput = 132,570 IOPS
Bandwidth = 10,270 MB/sec
Interconnect = 2,377 MB/sec
Single
Container
OLTP
CDB
CPU
PDB
IO
PDB
DSS
PDB
CRS
non-CDB

26. Cisco Confidential 26© 2013-2014 Cisco and/or its affiliates. All rights reserved.
TPM – 1,162,648
Throughput = 224,864 IOPS
Bandwidth = 10,356 MB/sec
Interconnect = 2,426 MB/sec
Multiple Containers vs Single Container with PDBs
CPU
DSS
Interconnect
Non-CDB
database
Interconnect
Non-CDB
database
TPM – 1,241,552
Throughput = 132,570 IOPS
Bandwidth = 10,270 MB/sec
Interconnect = 2,377 MB/sec