Database and application consolidation on oracle super cluster

1. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.1

2. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.2
Database and Application
Consolidation
on Oracle SuperCluster
Download this slide
http://ouo.io/63Zwq

3. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.3
Program
Agenda
 Introduction
 Enabling Technologies
 Consolidation Use Cases
 Recommendations
 Q&A

4. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.4
The following is intended to outline our general product
direction. It is intended for information purposes only, and
may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing
decisions.
The development, release, and timing of any features or
functionality described for Oracle’s products remain at the
sole discretion of Oracle.

5. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.5
Introduction

6. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.6
The Objective
Showcase the value of Oracle
SuperCluster for application +
database consolidation

7. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.7
Oracle Engineered Systems
Data Center Transformation Through Innovation
Consolidation Built
Oracle SuperCluster
Purpose Built
Database Appliance Exalytics
Big DataExalogicExadata

8. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.8
Exadata Exalogic
Exadata Storage
Database
Servers
ZFS NAS Storage
Application
Servers
Oracle SuperCluster
Oracle SuperCluster
Engineered System Building Blocks for General Purpose

9. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.9
• 2 * T5-8, each with
– 4 * SPARC T5 Processors (64 cores)
– 1 TB Memory
– 4 * InfiniBand HCAs (dual port)
– 4 * 10GbE NICs (dual port)
– 8 * 900GB SAS disks
• Storage
– 4 Exadata Storage Servers
– ZFS Storage Appliance (60TB disk and 4*73GB Logzillas)
• Switches
– 3 * InfiniBand 36 port switches
– Cisco 4948 GbE Management switch
Oracle SuperCluster T5-8
Half-Rack Configuration Details

10. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.10
• 2*T5-8, each with
– 8 * SPARC T5 Processors (128 cores)
– 2 TB Memory
– 8 * InfiniBand HCAs (dual port)
– 8 * 10GbE NICs (dual port)
– 8 * 900GB SAS disks
• Storage
– 8 Exadata Storage Servers
– ZFS Storage Appliance (60TB disk and 4*73GB Logzillas)
• Switches
– 3 * InfiniBand 36 port switches
– Cisco 4948 GbE Management switch
Oracle SuperCluster T5-8
Full-Rack Configuration Details

11. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.11
Oracle SuperCluster T5-8
Summary of Configurations
SSC T5-8 Half Rack SSC T5-8 Full Rack
T5-8 Nodes 2 2
T5 Compute Cores 128 (64 cores per node) 256 (128 cores per node)
T5 Memory 2TB (1TB per node) 4TB (2TB per node)
Exadata Storage
Servers
4 8
ZFS SA 60TB 60TB
~2.3x Compute capacity and 33% more Exadata Storage for both
Full Rack and Half Rack SSC T5-8 vs SSC T4-4

12. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.12
• Same Great Benefits
– Best platform for full-stack application deployments
– Best platform for consolidation
– Layered Optimized Virtualization
– Engineered to optimize Oracle software stack
Oracle SuperCluster T5-8
Configuration Overview

13. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.13
Oracle SuperCluster
Software Components
Any Oracle, ISV or customer application
for Solaris 10 or 11
Oracle Fusion Middleware, Exalogic Elastic Cloud,
or 3rd party middleware
Oracle 11gR2
Solaris 10 or Solaris 11
Oracle VM for SPARC
Exadata Storage Server Software

14. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.14
Oracle Solaris 11
Manage Both Virtual and Physical Infrastructure
Physical
Storage Pools
Cloud
Virtual Switch Virtual Switch
VNIC Virtual NIC VNIC
Logical Domain Logical Domain
Virtual Storage
Virtual
Networking
Virtual Servers
Oracle Solaris 11
Zone Zone Zone Zone

15. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.15
Isolated OS instance in
each logical (or virtual)
domain
Firmware-based hypervisor
leverages SPARC T5 architecture
Each logical domain runs in
dedicated CPU thread(s)
T-Series Server
Oracle Solaris 10
Oracle Solaris 11
Database Domain
Oracle Solaris 10
Oracle Solaris 11
Database Domain
Oracle Solaris 10
Oracle Solaris 11
Database Domain
Oracle Solaris 10
Oracle Solaris 11
Database Domain
App Domain App DomainApp Domain App Domain
SPARC Hypervisor
Domains can have direct
access to PCIe devices
Oracle VM Server for SPARC
Efficient Hypervisor-based Virtualization

16. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.16
Solaris Zones
• Namespace isolation: users, services,
network, etc.
• Minimal overhead and small footprint
• Independent, fast reboot
• Easy to clone/attach/detach
• Support for older releases through
Branded Zones
• Supported partitioning for Oracle
licensing purposes (capped zones)
Efficient Virtualization

17. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.17
Solaris Network Virtualization
• Allows testing and implementation of
complex network topologies
• Zones get exclusive IP stack by default
• Low-latency and high-bandwidth
communication between Zones
• Integrated firewall, load balancer and
router
• Integrated network resource control
• Bandwidth and Flow
Network infrastructure within the OS

18. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.18
Oracle Enterprise Manager Ops Center
Complete Management for Oracle Hardware, OS & Virtualization
• Data Center Discovery
• Virtualization Management
– Oracle, Zones, Containers
• Configuration Management
– Patch OS, Update Firmware,
Configuration Compliance
• Operating System Analytics
• Maintenance
– Health Checks, Remote Management,
Phone Home

19. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.19
App + DB Consolidation
Use Cases

20. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.20
Consolidation on ½ Rack Oracle SuperCluster
– Order Management & Payroll Management
 App running on Oracle Solaris 11 with HA
 DB on Oracle RAC 11gR2
– Legacy customer application
 App running on Oracle Solaris 10
 Standalone Oracle 10gR2 DB
– Web site deployment
 Web App(s) running on Oracle Solaris
 Standalone Oracle 11gR2 DB
Architecture Overview – A hypothetical scenario

21. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.21
Consolidation on ½ Rack Oracle SuperCluster
Architecture Overview – Hardware Infrastructure
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
Oracle SPARC T5-8 Node 1 Oracle SPARC T5-8 Node 2
Hypervisor Hypervisor
Hardware
CPU, Memory,
PCIe
Hardware
CPU, Memory,
PCIe
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
Oracle SPARC T5-8 server
- Built-in, no-cost virtualization simplifies server consolidation,
improves utilization, and reduces operational overhead
- High speed onboard encryption engines provide data security
4 processors
1 TB RAM
4 PCI Bus
4 processors
1 TB RAM
4 PCI Bus

22. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.22
Consolidation on ½ Rack Oracle SuperCluster
Architecture Overview – LDoms Creation
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
T5-8 Node 1 T5-8 Node 2
Hypervisor Hypervisor
DB Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
2 processor
512 GB RAM
2 PCI Bus
DB Domain App Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
Oracle VM Server for SPARC
- Low-level implementation via firmware
- Hard CPU and memory resource constraints
- Guest OS fault isolation
- Multiple OS versions and patch levels on the same physical host

23. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.23
Consolidation on ½ Rack Oracle SuperCluster
Architecture Overview – OS Setup
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
T5-8 Node 1 T5-8 Node 2
Hypervisor Hypervisor
DB Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 Processor
256 GB RAM
1 PCI Bus
2 Processors
512 GB RAM
2 PCI Bus
DB Domain App Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
Oracle
Solaris 11
Oracle
Solaris 11
Oracle Solaris 10 Oracle
Solaris 11
Oracle
Solaris 11
Oracle
Solaris 11
Oracle
Solaris 11
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
DB Zone App Zone
Oracle Solaris 11
- Binary compatibility with older releases
- Proven scalability record on large number of hardware threads
- Built-in virtualization with minimal overhead
- Efficient Packaging system
- Fine-grained security
- Fine-grained control of processor and network resources
Oracle Solaris Zones
- Efficient virtualization with minimal overhead
- Support for both Oracle Solaris 10 and Oracle Solaris 11.
- Branded Zones facilitate migration from older Solaris releases
- Delegated zones administration

24. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.24
Consolidation on ½ Rack Oracle SuperCluster
Architecture Overview – DB & Apps Installed
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
T5-8 Node 1 T5-8 Node 2
Hypervisor Hypervisor
DB Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
2 processors
512 GB RAM
2 PCI Bus
DB Domain App Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle Solaris 10
Oracle 10g R2
Single Instance
Legacy App
Sun Cluster
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle
Solaris 11
Oracle 11g R2
Single Instance
Oracle
Solaris 11
Web App
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
Sun ClusterDB Zone App Zone
Oracle Exadata storage servers
- Smart use of Flash accelerates database I/O and reduces log file latency
- Smart Scan provides the ability to offload database processing from compute nodes
- Hybrid Columnar Compression (HCC) reduces storage footprint per database
- IO Resource Manager (IORM) assigns I/O bandwidth based on database SLAs
Sun ZFS Storage Appliance provides performant, highly-available storage available
for both shared filesystem as well as a block storage

25. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.25
Architecture Overview – Cluster Setup
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
T5-8 Node 1 T5-8 Node 2
Hypervisor Hypervisor
DB Domain App Domain App Domain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
2 processor
512 GB RAM
2 PCI Bus
DB Domain App Domain App Domain App DOmain
Hardware
CPU, Memory,
PCIe
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle Solaris 10
Oracle 10g R2
Single Instance
Legacy App
Oracle Solaris
Cluster
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle
Solaris 11
Oracle 11g R2
Single Instance
Oracle
Solaris 11
Web App
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
Oracle Soalris
Cluster
DB Zone App Zone
Oracle Solaris Cluster provides application level HA while Oracle RAC provides HA
for the DB.
Consolidation on ½ Rack Oracle SuperCluster

26. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.26
Consolidation on ½ Rack Oracle SuperCluster
Architecture Overview – System Management
I n f i n i B a n d N e t w o r k
10 GbE Data Center Network
T5-8 Node 1 T5-8 Node 2
DB Domain App Domain App Domain
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
2 processors
512 GB RAM
2 PCI Bus
DB Domain App Domain App Domain App Domain
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processor
256 GB RAM
1 PCI Bus
1 processors
256 GB RAM
1 PCI Bus
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle Solaris 10
Oracle 10g R2
Single Instance
Legacy App
Oracle Solaris
Cluster
Oracle
Solaris 11
Oracle RAC
11g R2
Oracle
Solaris 11
Management
App
Oracle
Solaris 11
Oracle 11g R2
Single Instance
Oracle
Solaris 11
Web App
Sun ZFS Storage
Appliance
ZFS Storage
Appliance
ZFS Storage
Appliance
Oracle Exadata
Storage Servers
Oracle Soalris
Cluster
DB Zone App Zone
Oracle Enterprise Manager Ops Center 12c
- Integrated control of all assets
- Detailed Analytics
- One-stop for system-wide maintenance
- Elastic resource control based on policies
- Included at no additional cost

27. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.27
Oracle Optimized Solutions
 Designed, tested and fully
documented architectures
 Tuned for optimal
performance and
availability
 Detailed Technical and
Business Whitepapers

28. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.28
Oracle
RAC
Zone
Cluster
Zone
Cluster
T5-8 Node 1
Oracle Solaris 10
App Domain
Oracle Solaris 11
DB Domain
T5-8 Node 2
Oracle Solaris 10
App Domain
Oracle Solaris 11
DB Domain
InfiniBand Network
Optimized Solution for PeopleSoft HCM
 App Tier components deployed in Zones
 Over 1M net payments per hour (batch)
 Up to 8000 concurrent Self-Service
users (online)
 Sub-second average response times
 Application-level HA with Solaris Cluster
 Under 15 seconds to detect failure
 Service restored in less than 2 minutes

29. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.29
InfiniBand Network
Solaris 10
Node 1
DB Domain
App Domain
10 GbE Data Center Network
Oracle SuperCluster
 SAP Applications and Central
Services run on Solaris 10
domains
 HA for SAP Central Services
through Solaris Cluster
 Domain failure detected in 11
seconds
 Service recovered in less than
2 minutes
 4x faster time to deployment
compared to standalone
components
Node 2
Solaris 11
ZFS
Appliance
Oracle Exadata Storage Servers
Optimized Solution for SAP
Solaris 10
DB Domain
App Domain
Solaris 11
Node 2

30. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.30

31. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.31
 Solution under development
 BRM and TimesTen
deployed on Zones
 IB connectivity helps meet
low latency requirements
 Processors sets assign cpu
resources based on
business cycle
 Over 12M calls per hour
(online)
 Over 84M call records per
hour (batch)
Optimized Solution for BRM
T5-8 Node 1
Oracle Solaris 10
App Domain
Oracle Solaris 11
DB Domain
T5-8 Node 2
Oracle Solaris 10
App Domain
Oracle Solaris 11
DB Domain
InfiniBand Network
COMMUNICATIONS
Billing and Revenue
Management
COMMUNICATIONS
Billing and Revenue
Management
TIMESTEN
TIMESTEN

32. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.32
Consolidation
Recommendations

33. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.33
 Hardware pool (aka. Cloud Pool)
– Administrator / Policy managed clusters
 Consolidation density
– Over-subscription of upto 2 x cores using instance caging
 Storage
– Exadata storage supporting all its performance enhancing features
 Provisioning
– Supports native, embedded and templates-based provisioning
 Patching
– Supports in-place and out-of-place patching via rolling / non-rolling updates
Consolidation Enablers for Oracle SuperCluster

34. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.34
 Resource Management
– Prioritization and management of workloads / databases based on IORM &
DBRM policies
– Dynamically grow / shrink cloud pools
– QoS Management monitors workloads across entire system
 Disaster Recovery & High Availability
– High Availability via Oracle Solaris Cluster for applications and Oracle RAC for
Oracle DB
– Supports Oracle GoldenGate & Oracle Data Guard
Consolidation Enablers for Oracle SuperCluster

35. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.35
Engineered with Best Practices

36. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.36
DB Consolidation on Oracle SuperCluster
 Minimum 11gR2 Domain Configuration
– Must have at least 1 x IB HCA and 1 x 10Gbe physical interface
 Minimum Solaris 11 Zone Configuration
– 1-core (8 vcpus)
Considerations

37. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.37
DB Consolidation on Oracle SuperCluster
 Separate Oracle Grid Infrastructure per zone
 Separate Database ORACLE_HOME per zone
 Separate Diskgroups and Griddisks per zone
– Configure Database-scoped Security
Best Practice Summary for Isolation

38. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.38
DB Consolidation on Oracle SuperCluster
 Each domain has multiple
zones
 Use Solaris resource
manager to ensure one
database in a zone doesn’t
swamp another
 Can dynamically shift CPU
and memory resources
between Zones
DB Consolidation Example
Oracle

39. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.39
DB Consolidation on Oracle SuperCluster
 Configure Diskgroup sets per zone:
– DBFS_DG
 1GB minimum, required for Voting Disks
– RECO_DG
– DATA_DG
 Stripe across all storage cells
– Requires 3 * #zone griddisks
– Use Database-scoped security for Diskgroup isolation
ASM Deployment

40. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.40
DB Consolidation on Oracle SuperCluster
Storage Cells  High performance disks
– 12 * 600GB 15,000 RPM drives
– 9120GB capacity uncompressed triple
mirrored after ASM installed
– 9120GB / 22 (We have 22 x 2-Node RAC)
– ~414GB per 2-Node RAC cluster
 High capacity disks
– 12 * 3TB 7,200 RPM drives
– ~43TB / 22
– ~1.9TB per 2-Node RAC cluster
Capacity Analysis
4

41. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.41
In Summary

42. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.42
Oracle SuperCluster
 Built-in virtualization
 Choice of virtualization based on isolation requirements
 Fully tested and supported configurations
 Multiple OS & patch levels on the same physical host
 Legacy support with Branded Zones
 Fully integrated with Exadata and Exalogic stack
 Seamless system and patch management via Oracle Enterprise
Manager Ops Center 12c
Engineered with Best Practices

43. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.43
Tues July 23, 2013
12:15 - 13:15 GEN1683 Getting Started with Oracle Engineered Systems
14:15 - 15:15 CON1679 The Future of Oracle Exadata: Developments for OLTP, Warehousing, and
Consolidation
15:30 - 16:30 CON1682 The Business Case for Oracle Exalogic
Wed July 24, 2013
8:45 - 9:45 CON1659 Application Deployment and Consolidation Best Practices on Oracle Exadata
12:00 - 13:00 CON1685 Deploying Database and Application Cloud Services with Engineered Systems
14:00 - 15:00 CON1678 Deep Dive into SPARC SuperCluster: Engineered for Extreme Efficiency
15:15 - 16:15 CON1684 Oracle Engineered Systems: Optimizing Your Oracle Applications
17:15 - 18:15 CON1681 Oracle Exalogic 2.0: Ready-to-Deploy, Mission-Critical Private Cloud
Thurs July 25, 2013
9:00 - 10:00 CON1657 Optimizing Enterprise Applications with Oracle Engineered Systems
10:15 - 11:15 CON1680 Oracle Exadata Technical Deep Dive: Architecture and Internals
12:00 - 13:00 CON1658 Database and Application Consolidation on SPARC SuperCluster
Oracle SuperCluster Conference Sessions

44. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.44
Graphic Section Divider

45. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.45