Understanding Oracle RAC 12c Internals OOW13 [CON8806]

9/30/13

1

Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template1
Understanding
Oracle RAC 12c Internals
Markus Michalewicz, Mark Scardina
Director of Product Management

9/30/13

2

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
Agenda
§  Introduction
§  The New Features in Oracle Grid Infrastructure and RAC 12c
–  Flex ASM
–  Flex Cluster
§  Policy-Managed Databases
–  Improved HA Management
–  Dynamic Database Services
§  Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c
§  Summary and Q&A

9/30/13

3

The New Oracle RAC 12c
Oracle RAC 12c provides**:
1.  Better Business Continuity
and High Availability (HA)
2.  Agility and Scalability
3.  Cost-effective Workload Management
Using
§  Standardized and improved
deployment and management
§  A familiar and matured HA stack
Oracle Real Application Clusters (RAC)
Oracle Grid Infrastructure (GI):
Automatic Storage Management (ASM)
Oracle Clusterware (OCW)
Oracle Flex ASM
CRM

**Understanding Oracle RAC Internals 11g Rel. 2:
http://www.oracle.com/technetwork/products/clustering/overview/understandingrac11grel2internals-2010098.pdf
The Setup for this Session
What has been installed and configured so far?
vixen srvpool: frontofficecomet srvpool: backoffice
dancer srvpool: backofficedasher srvpool: frontoffice
OS OL 6.4 OS OL 6.4
OS OL 6.4OS OL 6.4
Oracle GI for a cluster Oracle GI for a cluster
Oracle GI for a clusterOracle GI for a cluster
Oracle RAC Oracle RAC
Oracle RACOracle RAC
raccdb1
raccdb1_2
raccdb1_1
[RAC]> srvctl status database -d raccdb1
Instance raccdb1_1 is running on node comet
Instance raccdb1_2 is running on node dancer
Instance raccdb1_3 is running on node vixen
Instance raccdb1_4 is running on node dasher
raccdb1_4
raccdb1_3
§  4-node cluster
§  OL 6.4 UEK
§  Oracle Grid Infrastructure 12c
–  Per node installation
§  Oracle RAC Database Home 12c
–  Shared installation on ACFS
§  One Oracle Multitenant Container Database
–  DB name: raccdb1
–  Deployed as a Policy-Managed Database

9/30/13

4

New Memory Requirements
Not an Oracle RAC internal – just something to mention
[GRID]> srvctl status asm
ASM is running on comet,dasher,dancer
[GRID]> srvctl status mgmtdb
Database is enabled
Instance -MGMTDB is running on node dasher
[GRID]> ps -ef |grep pmon
grid 3227 1 0 13:54 ? 00:00:02 asm_pmon_+ASM1
grid 3868 1 0 13:55 ? 00:00:01 mdb_pmon_-MGMTDB
grid 7364 1 0 13:57 ? 00:00:01 apx_pmon_+APX1
oracle 7917 1 0 13:57 ? 00:00:02 ora_pmon_raccdb1_4
[RAC]> df -h
Filesystem Size Used Avail Use% Mounted on
...
tmpfs 1.7G 1.3G 475M 73% /dev/shm
...
Physical memory requirements
with Oracle RAC 12c: 4GB
Agenda
–  Flex ASM
–  Flex Cluster

9/30/13

5

Oracle ASM 12c – Overview
Oracle ASM 12c Standard Deployment
ASM Cluster Pool of Storage
Disk Group BDisk Group AShared Disk
Groups
Wide File Striping
One to One
Mapping of ASM
Instances to
Servers
ASM Instance
Database Instance
ASM Disk
RAC Cluster
Node4Node3Node2Node1 Node5ASM ASM ASM ASM ASM
ASM Instance
Database Instance
DBA DBA DBCDBB DBBDBB
Introducing Oracle Flex ASM
Removal of One to One Mapping and HA
Groups
Wide File Striping
Databases share
ASM instances
ASM Instance
Database Instance
ASM Disk
RAC Cluster
Node5Node4Node3Node2Node1
Node5
runs as
ASM Client
to Node4
Node1
runs as
ASM Client
to Node2
Node1
runs as
ASM Client
to Node4
Node2
runs as
ASM Client
to Node3
ASM ASM ASM
ASM Instance
DBA DBA DBCDBB DBBDBB

9/30/13

6

Supporting Pre-Oracle 12c Databases
Pre-Oracle 12c Databases require a local ASM instance
Groups
Wide File Striping
Databases share
ASM instances
ASM Instance
Database Instance
ASM Disk
RAC Cluster
Node5Node4Node3Node2Node1 ASM ASM ASM
DBA DBA DBC
ASM ASM
11g
DB
11g
DB
DBB DBBDBB
Oracle Flex ASM
Configuration details
§  Flex ASM introduces new local resources:
–  At least one ASM listener
–  One “proxy_advm” (per node)
§  Used for ACFS access to Flex ASM instances
§  Connections from a database instance to an ASM
instance are based on SQLnet using listeners.
–  The listener directs the connection to the least loaded
ASM instance based on the load metric it maintains.
–  The connection details are fetched from CSS global data
–  The ASM instance to which the database instance
connects to is listed in the database alert log:
§  NOTE: ASMB connected to ASM instance
+ASM1 (Flex mode; client id 0x10004)
–  The userid and password supplied are also managed
automatically. They are supplied while establishing the
session, not while connecting.
[GRID]> crsctl stat res -t
-------------------------------------------------------------------------------
Name Target State Server State details
-------------------------------------------------------------------------------
Local Resources
-------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE comet STABLE
ONLINE ONLINE dancer STABLE
ONLINE ONLINE dasher STABLE
ONLINE ONLINE vixen STABLE
...
ora.DATA.dg
OFFLINE OFFLINE vixen STABLE
...
ora.INFRA.dg
...
ora.proxy_advm

9/30/13

7

Oracle Flex ASM
Administering Flex ASM
§  Check ASM mode using asmcmd
§  Check where ASM instances are running using srvctl
§  Check the configuration of ASM using srvctl
§  Modify the ASM configuration using srvctl
§  Cluster ASM listener:
–  Flex ASM can have its own network or overlap with the
private interconnect network used by RAC DB instances
–  It is recommended to configure a full overlap of the private
interconnect and the ASM network in current use cases:
[GRID]> asmcmd
ASMCMD> showclustermode
ASM cluster : Flex mode enabled
[GRID]> srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +INFRA/orapwASM
ASM listener: LISTENER
ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM,ASMNET2LSNR_ASM
[GRID]> srvctl modify asm -count all
[GRID]> srvctl start asm
ASM is running on comet,vixen,dasher,dancer
Test scenarios attached
Oracle Flex ASM
Some test scenarios…
§  A failure of an ASM instance is without effect for the
databases using ASM disk groups (here: -MGMTDB)
–  If an ASM instance is running on all nodes (count all)
the ASM instance is restarted upon failure
–  If an ASM instance is running on a subset of nodes
the ASM instance might be restarted on another node
–  The client using the ASM instance may change
§  A failure of an ASM instance is without effect for
the APX (ADVM proxy) running on the same node
§  A failure of the APX will lead to a restart of the database
instance (on the node) that uses an ACFS file system.
§  More information:
–  http://docs.oracle.com/cd/E16655_01/server.121/e17612/
asminst.htm#OSTMG95329
grid 4093 1 0 11:35 ? 00:00:00 asm_pmon_+ASM1
grid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1
[GRID]> kill -9 4093
grid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1
grid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1
grid 5748 1 0 11:39 ? 00:00:00 asm_pmon_+ASM1
grid 5793 19013 0 11:39 pts/1 00:00:00 grep pmon
[GRID]> asmcmd lsct data
DB_Name Status Software_Version Compatible_version Instance_Name Disk_Group
+APX CONNECTED 12.1.0.1.0 12.0.0.0.0 +APX1 DATA
[GRID]> ssh vixen /u01/app/12.1.0/grid/bin/asmcmd lsct data
DB_Name Status Software_Version Compatible_version Instance_Name Disk_Group

9/30/13

8

New in Oracle Grid Infrastructure 12c
Oracle Flex Cluster for better resource management
DBA
DBA
DBA
DBB
DBB
Hub Nodes
Leaf Nodes
§  Oracle Flex Cluster:
–  The new standard for flexible
resource management in the cluster
§  Use it as demand requires it
§  Requires Oracle Flex ASM to be enabled
§  Enables policy-managed clusters.
–  Efficient High Availability and
resource management across tiers
§  Lightweight cluster stack on leaf nodes
–  Benefit from virtualization and
physical deployments in one cluster
Oracle Flex Cluster
Some configuration details
§  Oracle Flex Cluster
–  Will be the basis for more uses cases
in future releases (for resource mgmt)
–  Can consist of only HUB-nodes
–  If a Leaf-node has access to shared storage,
it can perform a role change to a HUB-node
–  A Standard Cluster can be converted to a
Flex Cluster at any time after the installation
–  Once a cluster is configured as a Flex Cluster
this configuration is irreversible without a re-
configuration of the cluster.
§  More Information:
–  http://docs.oracle.com/cd/E16655_01/rac.121/e17886/
bigcluster.htm#CWADD92560
[GRID]> crsctl get cluster mode config
Cluster is configured as "standard"
[GRID]> crsctl get cluster mode status
Cluster is running in "standard" mode
[GRID]> olsnodes -s -a -t
dasher Active Hub Unpinned
vixen Active Hub Unpinned
dancer Active Hub Unpinned
comet Active Hub Unpinned
•  Not a Flex Cluster yet.
•  An all-Hub Flex Cluster is
basically a Standard Cluster
Example attached

9/30/13

9

Oracle Flex Cluster
Converting to Flex Cluster
bigcluster.htm#CHDJGBII
§  Flex Cluster implementations require GNS
§  Converting a Flex Cluster requires a cluster restart
–  Use “crsctl stop crs” followed by “crsctl start crs” as root
–  Hence the difference between “status” and “config”
–  Nodes can be restarted in a rolling fashion:
[GRID]> srvctl config gns
PRKF-1110 : Neither GNS server nor GNS client is configured on
this cluster
[GRID]> more /etc/hosts |grep gns
10.1.1.99 mygns mygns.localdomain
#Change to root
[GRID]> su
Password:
[GRID]> srvctl add gns -vip mygns
[GRID]> srvctl start gns
[GRID]> crsctl set cluster mode flex
Cluster mode set to "flex »
[GRID]> crsctl get cluster mode config
Cluster is configured as "flex"
[GRID]> date ; hostname ;
crsctl get cluster mode status
Sat Sep 14 14:43:45 PDT 2013
dasher
Cluster is running in "flex" mode
[GRID]> date ; hostname ;
crsctl get cluster mode status
Sat Sep 14 14:43:37 PDT 2013
dancer
Oracle Flex Cluster
Standard to Flex Cluster conversion – what has changed?
vixen srvpool: frontofficecomet srvpool: backoffice
Oracle GI | HUB Oracle GI | HUB
Oracle GI | HUBOracle GI | HUB
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
Cluster is running in "flex" mode
[GRID]> olsnodes -a -s -t
dasher Active Hub Unpinned
vixen Active Hub Unpinned
dancer Active Hub Unpinned
comet Active Hub Unpinned
§  Nothing at first glance…
–  An all-Hub Flex Cluster behaves
like a Standard Cluster
–  When converting from Standard Cluster
all nodes are converted into HUB-nodes

9/30/13

10

Oracle Flex Cluster
Flexibility and High Availability as demand requires it
§  Flex Cluster utilizes two types of cluster nodes:
–  Hub nodes
§  Traditional nodes, tightly coupled via network and
storage, used for running IO intensive workload
–  Leaf nodes
§  A new type of node that runs a lighter weight stack
§  Maintain their own failure and heartbeat settings
§  Do not require direct shared storage connectivity
§  Used for processing oriented (application) workload
Hub Nodes
Leaf Nodes
Oracle Flex Cluster
Changing node roles – server vixen becomes a Leaf node
bigcluster.htm#BABHCEAE
[GRID]> hostname
vixen
[GRID]> srvctl config vip -node vixen
VIP exists: network number 1, hosting node vixen
VIP Name: vixen-vip
VIP IPv4 Address: 10.1.1.33
VIP IPv6 Address:
[GRID]> crsctl get node role config
Node 'vixen' configured role is 'hub'
#Change to root …
[GRID]> su
Password:
[GRID]> crsctl set node role leaf
CRS-4408: Node 'vixen' configured role successfully changed; restart
Oracle High Availability Services for new role to take effect.
[GRID]> crsctl stop crs
[GRID]> crsctl start crs
[GRID]> crsctl get node role config
Node 'vixen' configured role is 'leaf'
VIP Name: vixen-vip
VIP IPv6 Address:

9/30/13

11

Oracle Flex Cluster
Node role change to Leaf – what has changed?
vixen srvpool: FREEcomet srvpool: backoffice
Oracle GI | LEAFOracle GI | HUB
raccdb1
raccdb1_4 raccdb1_2
raccdb1_1
[RAC]> srvctl status server -servers vixen -detail
Server active pools: Free
Server name: vixen
Server state: ONLINE
Server state details:
§  Server vixen has converted to a Leaf node
§  Leaf nodes do not host DB instances in 12.1
§  Server vixen moved to Server Pool “FREE” as
Server Pool “frontoffice” is a database pool
–  Note: all nodes in this example have access
to shared storage, which means only the node
role changes – storage access is preserved.
Example attached
Oracle Flex Cluster
Revert role change – server vixen becomes a HUB again
comet srvpool: backoffice
Oracle GI | HUB
Oracle RAC
raccdb1
raccdb1_4 raccdb1_2
raccdb1_1
vixen srvpool: frontoffice
Oracle GI | HUB
Oracle RAC
raccdb1_3
#As root
[GRID]> crsctl set node role hub
CRS-4408: Node 'vixen' configured role successfully changed;
restart Oracle High Availability Services for new role to take
effect.
[GRID]> crsctl stop crs
[GRID]> crsctl start crs

9/30/13

12

Agenda
–  Flex ASM
–  Flex Cluster
Policy-Managed Databases with Oracle RAC 12c
Adding another dimension – Database Services
Oracle GI | HUB
Oracle RAC
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
[GRID]> srvctl status serverpool
Server pool name: frontoffice
Active Servers count: 2
Server pool name: backoffice
[RAC]> srvctl status service -d raccdb1
Service crmsvc is running on nodes: dasher,vixen
Service hrsvc is running on nodes: comet,dancer
§  Database Services
–  Ensure that workload is hosted in the respective
server pool with the cardinality defined as part of
the policy definition.
Oracle GI | HUB
Oracle RAC
raccdb1_1

9/30/13

13

Improved HA Management - Startup
raccdb1
[GRID]> srvctl config serverpool
Importance: 10, Min: 2, Max 2
Server pool name: Free
Importance: 0, Min: 0, Max -1
§  Defining a Service Startup Order
–  Ensure services are started in specified groups
and specified order.
–  Leverages Serverpool Min and Importance properties
dasher srvpool: frontoffice
Oracle GI | HUB
Oracle RAC
raccdb1_4
1st
Oracle GI | HUB
Oracle RAC
raccdb1_3
2nd
Oracle GI | HUB
Oracle RAC
raccdb1_1
4th
dancer srvpool: backoffice
Oracle GI | HUB
Oracle RAC
raccdb1_2
3rd
Oracle GI | HUB
Oracle RAC
raccdb1_2
dancer srvpool: frontoffice
Oracle GI | HUB
Oracle RAC
raccdb1_2
Improved HA Management – New Failover Strategy
raccdb1
[GRID]> srvctl status serverpool
Oracle GI | HUB
Oracle RAC
raccdb1_4
Oracle GI | HUB
Oracle RAC
raccdb1_3
Oracle GI | HUB
Oracle RAC
raccdb1_1
§  Servers “Move” to Replace Failed node
–  Protects against cascade failures due to load
–  Ensures workload isolation between pools
–  Less important workloads shutdown transactionally

9/30/13

14

Oracle GI | HUB
Oracle RAC
raccdb1_2
Oracle GI | HUB
Oracle RAC
raccdb1_2
Improved HA Management – Manage Last Service Standing
raccdb1
[RAC]> srvctl config serverpool
[RAC]> srvctl status service –db
Service crmsrv is running on nodes comet,vixen
Service hrsvc is not running
Oracle GI | HUB
Oracle RAC
raccdb1_4
Oracle GI | HUB
Oracle RAC
raccdb1_3
Oracle GI | HUB
Oracle RAC
raccdb1_1
§  Business Critical Services survive multiple failures
–  Most important pool always gets the servers
–  Services preserved across multiple failures
–  Less important workloads shutdown transactionally
comet srvpool: frontoffice
Oracle GI | HUB
Oracle RAC
raccdb1_1
Improved HA Management – Dynamic Provisioning
Oracle GI | HUB
Oracle RAC
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
[GRID]> srvctl modify serverpool
–serverpool backoffice –max 1
[RAC]> srvctl config serverpool
[RAC]> srvctl status service –db
Service crmsrv is running on nodes dasher,dancer,vixen
Service hrsvc is not running
Oracle GI | HUB
Oracle RAC
raccdb1_1
§  Add Servers Just-In Time to meet demand
–  Server Pools sized via Min and Max properties
–  Dynamically controlled by QoS Management
–  Planned control via Clusterware or QoSM Policies
Oracle GI | HUB
Oracle RAC
raccdb1_2

9/30/13

15

Agenda
–  Flex ASM
–  Flex Cluster
Policy-Management with Oracle RAC 12c
Support for Multiple Policies tracking business objectives
–  http://docs.oracle.com/cd/E16655_01/rac.121/e17886/pbmgmt.htm
§  New in Oracle Grid Infrastructure 12c
–  Server Categories
§  Server Categories use server attributes to allow
for an active use of differently sized servers
–  Policy Sets
§  Policy Sets allow for dynamic adjustment
to demand changes in an atomic transaction
Server Categories
•  NAME
•  ACL
•  EXPRESSION
•  …
Server Attributes
•  NAME
•  MEMORY_SIZE

•  CPU_COUNT
•  CPU_CLOCK_RATE

•  CPU_HYPERTHREADING

•  CPU_EQUIVALENCY
• …

[GRID]> crsctl modify policyset –attr
"LAST_ACTIVATED_POLICY=NightTime“
CRS-2673: Attempting to stop
'ora.raccdb1.crmsvc.svc' on 'comet'
CRS-2673: Attempting to stop
'ora.raccdb1.crmsvc.svc' on 'dancer'
CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on
'comet' succeeded
CRS-2673: Attempting to start
'ora.raccdb1.backup.svc' on 'comet'
CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on
'dancer' succeeded
CRS-2672: Attempting to start
'ora.raccdb1.hrsvc.svc' on 'dancer‘
…
QoS Management
OR
Oracle Clusterware - CRSCTL

9/30/13

16

Setting up policy sets – Provision Server Pools and creating a PolicySet
§  Add another server pool “backup”
§  Set up policy set with 3 server pools & 3 policies as follows:
–  DayTime:
§  frontoffice uses three servers (MIN_SIZE=3)
§  backoffice uses one server (MIN_SIZE=1)
§  backup does not run during daytime (MIN_SIZE=0)
–  NightTime:
§  frontoffice uses one server (MIN_SIZE=1)
§  backoffice uses two servers (MIN_SIZE=2)
§  backup uses only one server (MIN_SIZE=1)
–  Weekend:
§  frontoffice uses one server (MIN_SIZE=1)
§  backoffice uses one server (MIN_SIZE=1)
§  backup uses two servers (MIN_SIZE=2)
[GRID]> srvctl add serverpool
–serverpool backup –min 0 –max 2 –importance 20
[GRID] srvctl status serverpool
Server pool name: backup
[GRID]> crsctl modify policyset
-file /u01/app/…/policyset.txt
Using Policy Sets – part 1: Activate a policy
§  Activate the policy for the weekend
[GRID]> crsctl modify policyset
-attr "LAST_ACTIVATED_POLICY=Weekend"
Server pool name: Free
Importance: 0, Min: 0, Max: -1
Category:
Candidate server names:
Server pool name: Generic
Importance: 0, Min: 0, Max: -1
Category:
Importance: 0, Min: 1, Max: 1
Category:
Category:
Category:

9/30/13

17

Using Policy Sets – part 2: check the result
vixen srvpool: frontofficecomet srvpool: backup
dancer srvpool: backupdasher srvpool: backoffice
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
Service backup is running on nodes: comet,dancer
Service crmsvc is running on nodes: vixen
Service hrsvc is running on nodes: dasher
§  With an activated weekend policy
Adding Server Categories to the picture – part 1
§  Assume you have 2 servers that have better IO
–  Use these servers for backups whenever possible
§  Here comet and dancer have better IO by definition
§  What you need to do:
–  Set up a server category that identifies the servers
–  Add the use of the server category to the server pool
§  Define the server pools that utilize the category
and during which policy activation it shall be used.
§  You need to restart the cluster stack on
the servers that you modify in this fashion
[GRID]> su
Password:
[GRID]> crsctl set server label IOplus
...
#On dancer
[GRID]> crsctl set server label Ioplus
[GRID]> crsctl get server label
CRS-4972: Current SERVER_LABEL parameter value is Ioplus
[GRID]> crsctl status server comet dancer –f
Comet Dancer
NAME=comet
MEMORY_SIZE=3338
CPU_COUNT=1
CPU_CLOCK_RATE=2132
CPU_HYPERTHREADING=0
CPU_EQUIVALENCY=1000
DEPLOYMENT=other
CONFIGURED_CSS_ROLE=hub
RESOURCE_USE_ENABLED=1
SERVER_LABEL=IOplus
...
NAME=dancer
MEMORY_SIZE=3338
CPU_COUNT=1
CPU_CLOCK_RATE=2247
CPU_HYPERTHREADING=0
CPU_EQUIVALENCY=1000
DEPLOYMENT=other
CONFIGURED_CSS_ROLE=hub
RESOURCE_USE_ENABLED=1
SERVER_LABEL=IOplus
...

9/30/13

18

Adding Server Categories to the picture – part 2
§  What you need to do:
–  Set up a server category that identifies the servers -
done
–  Add the use of the server category to the server pool
§  Define the server pools that utilize the category
and during which policy activation it shall be used
§  Note: Add the server category usage to the
server pool(s) that require it & specify the
policies that should be altered accordingly
–  In this example the backup server pool is the one
that will benefit the most and the policies will only
start this server pool as required.
[GRID]> crsctl add category moreIO
–attr "EXPRESSION='SERVER_LABEL co IOplus'"
[GRID]> crsctl status category moreIO
NAME=moreIO
ACL=owner:grid:rwx,pgrp:oinstall:rwx,other::r--
ACTIVE_CSS_ROLE=hub
EXPRESSION=(SERVER_LABEL co IOplus)
[GRID]> crsctl modify serverpool ora.backup
-attr "SERVER_CATEGORY=moreIO" -all_policies
Adding Server Categories to the picture – comparison
Before After
[GRID]> crsctl status policy DayTime
POLICY
NAME=DayTime
DESCRIPTION=
SERVERPOOL
NAME=Free
...
SERVERPOOL
NAME=ora.backoffice
IMPORTANCE=0
MAX_SIZE=1
MIN_SIZE=1
SERVER_CATEGORY=
SERVER_NAMES=
SERVERPOOL
NAME=ora.backup
IMPORTANCE=0
MAX_SIZE=0
MIN_SIZE=0
SERVER_CATEGORY=
SERVER_NAMES=
SERVERPOOL
NAME=ora.frontoffice
IMPORTANCE=0
MAX_SIZE=3
MIN_SIZE=3
SERVER_CATEGORY=
SERVER_NAMES=
[GRID]> crsctl status policy DayTime
POLICY
NAME=DayTime
DESCRIPTION=
SERVERPOOL
NAME=Free
...
SERVERPOOL
NAME=ora.backoffice
IMPORTANCE=0
MAX_SIZE=1
MIN_SIZE=1
SERVER_CATEGORY=
SERVER_NAMES=
SERVERPOOL
NAME=ora.backup
IMPORTANCE=0
MAX_SIZE=0
MIN_SIZE=0
SERVER_CATEGORY=moreIO
SERVER_NAMES=
SERVERPOOL
NAME=ora.frontoffice
IMPORTANCE=0
MAX_SIZE=3
MIN_SIZE=3
SERVER_CATEGORY=
SERVER_NAMES=

9/30/13

19

Using Policy Sets and the effect of Server Categories
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
vixen srvpool: backofficecomet srvpool: frontoffice
dancer srvpool: frontofficedasher srvpool: frontoffice
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
[GRID]> crsctl modify policyset –attr
"LAST_ACTIVATED_POLICY=DayTime“
[GRID] srvctl status serverpool
Service backup is not running.
Service crmsvc is running on nodes: dasher,vixen,comet
Service hrsvc is running on nodes: dancer
Using Policy Sets means changing policies on a push of a button
raccdb1
Oracle GI | HUB
Oracle RAC
raccdb1_3
Oracle GI | HUB
Oracle RAC
raccdb1_4
comet srvpool: backup
Oracle GI | HUB
Oracle RAC
raccdb1_1
dancer srvpool: backup
Oracle GI | HUB
Oracle RAC
raccdb1_2
Oracle GI | HUB
Oracle RAC
raccdb1_2
comet srvpool: frontoffice
Oracle GI | HUB
Oracle RAC
raccdb1_1

9/30/13

20

Using Policy Sets – changing policies on the push of a button is fast …
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
vixen srvpool: backofficecomet srvpool: frontoffice
dancer srvpool: frontofficedasher srvpool: frontoffice
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
[GRID]> date; crsctl modify policyset -attr
"LAST_ACTIVATED_POLICY=DayTime"; date
Mon Sep 16 19:26:42 PDT 2013
CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'dancer'
CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'comet'
CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'dancer' succeeded
CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'comet' succeeded
CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'dancer'
CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'comet'
CRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'dancer' succeeded
CRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'comet' succeeded
Mon Sep 16 19:26:43 PDT 2013
è Time to execute: 1 second!
What-If evaluation of policy changes
Service crmsvc is running on nodes: dancer,vixen,dasher
Service hrsvc is running on nodes: comet
[GRID]> crsctl eval activate policy Weekend
Stage Group 1:
-------------------------------------------------------------------------------
Stage Number Required Action
-------------------------------------------------------------------------------
1 Y Server 'comet' will be moved from pools
[ora.frontoffice] to pools [ora.backup]
Y Server 'dancer' will be moved from pools
[ora.frontoffice] to pools [ora.backup]
Y Resource 'ora.raccdb1.crmsvc.svc' (1/1) will be
in state [OFFLINE]
Y Resource 'ora.raccdb1.crmsvc.svc' (2/1) will be
in state [OFFLINE]
2 Y Resource 'ora.raccdb1.backup.svc' (1/1) will be
in state [ONLINE|INTERMEDIATE] on server[comet]
Y Resource 'ora.raccdb1.backup.svc' (2/1) will be
in state [ONLINE|INTERMEDIATE] on server
[dancer]
vixen srvpool: frontofficecomet srvpool: frontoffice
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1

9/30/13

21

What-If with Oracle RAC 12c
What-If evaluation of policy changes – in various levels
[GRID]> crsctl eval activate policy Weekend -admin -l 'serverpools'
NAME = Free
ACTIVE_SERVERS =
NAME = Generic
ACTIVE_SERVERS =
NAME = ora.backoffice
ACTIVE_SERVERS = vixen
NAME = ora.backup
ACTIVE_SERVERS = comet dancer
NAME = ora.frontoffice
ACTIVE_SERVERS = dasher
[GRID]> crsctl eval activate policy Weekend -admin -l 'resources'
--------------------------------------------------------------------------------
Name Target State Server Effect
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.mgmtdb
1 ONLINE ONLINE dasher
ora.raccdb1.backup.svc
1 ONLINE ONLINE comet Started
2 ONLINE ONLINE dancer Started
ora.raccdb1.crmsvc.svc
1 ONLINE OFFLINE Stopped
2 ONLINE OFFLINE Stopped
ora.raccdb1.db
1 ONLINE ONLINE comet
2 ONLINE ONLINE dancer
3 ONLINE ONLINE vixen
ora.raccdb1.hrsvc.svc
1 ONLINE ONLINE vixen
2 ONLINE OFFLINE
--------------------------------------------------------------------------------
Information on each step on the way
Before After
[GRID]> crsctl modify policyset -attr
"LAST_ACTIVATED_POLICY=NightTime“
CRS-2673: Attempting to stop 'ora.raccdb1.crmsvc.svc' on 'dancer'
CRS-2673: Attempting to stop 'ora.raccdb1.crmsvc.svc' on 'comet'
CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on 'dancer' succeeded
CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on 'comet' succeeded
CRS-2672: Attempting to start 'ora.raccdb1.backup.svc' on 'dancer'
CRS-2672: Attempting to start 'ora.raccdb1.backup.svc' on 'comet'
CRS-2676: Start of 'ora.raccdb1.backup.svc' on 'dancer' succeeded
CRS-2676: Start of 'ora.raccdb1.backup.svc' on 'comet' succeeded
[GRID]> crsctl modify policyset -attr
"LAST_ACTIVATED_POLICY=DayTime"
CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'dancer'
CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'comet'
CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'dancer' succeeded
CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'comet' succeeded
CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'dancer'
CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'comet'
CRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'dancer' succeeded
CRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'comet' succeeded

9/30/13

22

Agenda
–  Flex ASM
–  Flex Cluster
Multiple Subnet Support in Oracle RAC 12c
Preparing for multiple subnet usage – part 1
§  Multiple subnet support was introduced
with Oracle RAC 11g Rel. 2
–  Using multiple subnets per cluster can come in
handy for various reasons – backups for example.
vixencomet
ora.net1.networkora.net1.network
[GRID]> crsctl stat res ora.net1.network -t
------------------------------------------------------------------------
------------------------------------------------------------------------
Local Resources
------------------------------------------------------------------------
ora.net1.network
------------------------------------------------------------------------
[GRID]> oifcfg iflist
eth0 10.1.1.0
eth1 192.168.7.0
eth1 169.254.128.0
eth2 172.149.2.0
eth2 169.254.0.0
eth3 10.2.2.0
[GRID]> oifcfg getif
eth0 10.1.1.0 global public
eth1 192.168.7.0 global cluster_interconnect,asm
One more physical interface
that can be used for public or
private communication.
Private communication already
uses multiple subnets.
Example attached

9/30/13

23

Preparing for multiple subnet usage – part 2
§  Multiple subnet support can be
activated in two alternative ways:
1.  Defining interface name and subnet identifier
2.  Defining only a subnet identifier (interface: “*”)
§  Using approach one allows for more consistent
management and will be used in this example.
[GRID]> oifcfg setif -global eth3/10.2.2.0:public
[GRID]> oifcfg setif -global "*"/10.2.2.0:public
* 10.2.2.0 global public
PRIF-29: Warning: wildcard in network parameters can
cause mismatch among GPnP profile, OCR, and system
Alternative configuration:
Adding second network resource to support multiple public subnets
vixencomet
ora.net1.network:10.1.1.0ora.net1.network:10.1.1.0
[GRID]> su
Password:
[GRID]> srvctl add network -netnum 2
-subnet 10.2.2.0/255.255.255.0
[GRID]> exit
[GRID]> crsctl stat res ora.net2.network -t
------------------------------------------------------------------------
------------------------------------------------------------------------
Local Resources
------------------------------------------------------------------------
ora.net2.network
OFFLINE OFFLINE comet STABLE
OFFLINE OFFLINE dancer STABLE
OFFLINE OFFLINE dasher STABLE
------------------------------------------------------------------------
§  The preparation:
–  Prepare the physical connectivity – done
–  Prepare the stack as shown
§  Adding the network is one command
§  VIPs need to be added separately.

9/30/13

24

Adding VIPs on second network to support multiple public subnets
§  Adding node VIPs to second subnet:
[GRID]> more /etc/hosts
...
10.2.2.30 dasher-vip2 dasher-vip2.localdomain
10.2.2.31 dancer-vip2 dancer-vip2.localdomain
10.2.2.32 comet-vip2 comet-vip2.localdomain
10.2.2.33 vixen-vip2 vixen-vip2.localdomain
vixencomet
[GRID]> srvctl config vip -node comet
VIP exists: network number 1, hosting node comet
VIP Name: comet-vip
VIP IPv6 Address:
VIP Name: vixen-vip
VIP IPv6 Address:
[GRID]> su
Password:
[GRID]> srvctl add vip -node comet -k 2
-A comet-vip2/255.255.255.0
[GRID]> srvctl add vip -node vixen -k 2
-A vixen-vip2/255.255.255.0
[GRID]> srvctl add vip -node dancer -k 2
-A dancer-vip2/255.255.255.0
[GRID]> srvctl add vip -node dasher -k 2
-A dasher-vip2/255.255.255.0
vixen-vip / 10.1.1.33comet-vip / 10.1.1.32
vixen-vip2 / 10.2.2.33comet-vip2 / 10.2.2.32
Complete Example attached
Start using the second subnet VIPs
vixencomet
#Example only – need to check for all nodes
VIP Name: vixen-vip
VIP IPv6 Address:
VIP Name: vixen-vip2
VIP IPv6 Address:
[GRID]> srvctl status vip -node vixen
VIP vixen-vip is enabled
VIP vixen-vip is running on node: vixen
VIP vixen-vip2 is enabled
VIP vixen-vip2 is not running
[GRID]> srvctl start vip -node vixen
PRKO-2420 : VIP is already started on node(s): vixen
[GRID]> srvctl start vip -vip vixen-vip2
[GRID]> srvctl start vip -vip comet-vip2
[GRID]> srvctl start vip -vip dancer-vip2
[GRID]> srvctl start vip -vip dasher-vip2
vixen-vip / 10.1.1.33comet-vip / 10.1.1.32
vixen-vip2 / 10.2.2.33comet-vip2 / 10.2.2.32
§  Once VIPs are added verify the setup
§  Start the VIPs in order to use them
§  Note: you need to specify the VIP name for the
VIPs on the second subnet to be started, as the
default assumes the “default node VIP”.

9/30/13

25

Listeners in Oracle RAC 12c
The default listener is available by default
vixencomet
[GRID]> srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s):
comet,vixen,dasher,dancer
[GRID]> srvctl config listener
Name: LISTENER
Network: 1, Owner: grid
Home: <CRS home>
End points: TCP:1521
LISTENER: 10.1.1.13
and 10.1.1.33
LISTENER: 10.1.1.10
and 10.1.1.30
§  Various listeners are used in Oracle RAC 12c
§  These are all listeners – used in a certain way!
§  The simplest one is the default listener
§  A listener depends on a network
§  A listener will listen on the node and VIP IP
The default listener FAQ
vixencomet
[GRID]> srvctl modify listener -h
Modifies the configuration for the listener.
Usage: srvctl modify listener [-listener <lsnr_name>]
[-oraclehome <path>] [-endpoints "[TCP:]<port>[, ...][/
IPC:<key>][/NMP:<pipe_name>][/TCPS:<s_port>] [/SDP:<port>]"] [-
user <oracle_user>] [-netnum <net_num>]
#Example – adding another listener on another port on net 1
[GRID]> srvctl add listener -listener LSNR2NET1 -netnum 1
[GRID]> srvctl config listener -listener LSNR2NET1
Name: LSNR2NET1
Home: <CRS home>
[GRID]> srvctl status listener -listener LSNR2NET1
Listener LSNR2NET1 is enabled
Listener LSNR2NET1 is not running
[GRID]> srvctl start listener -listener LSNR2NET1
Listener LSNR2NET1 is running on node(s):
§  Can you define another port? - YES
§  Can you define another owner? – YES
§  Can you define another home? – YES, if needed.
§  Can I add another listener on another port? - YES
LISTENERLISTENER
LSNR2NET1LSNR2NET1
Example attached

9/30/13

26

The default listener FAQ
vixencomet
[GRID]> srvctl modify listener -listener LSNR2NET1
-endpoints "TCP:1528«
[GRID]> lsnrctl status LSNR2NET1
LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 16-SEP-2013 19:34:13
Copyright (c) 1991, 2013, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LSNR2NET1)))
STATUS of the LISTENER
------------------------
Alias LSNR2NET1
Version TNSLSNR for Linux: Version 12.1.0.1.0 - Production
Start Date 16-SEP-2013 16:35:34
Uptime 0 days 2 hr. 58 min. 39 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/12.1.0/grid/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/dasher/lsnr2net1/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LSNR2NET1)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.10)(PORT=1528)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.30)(PORT=1528)))
Services Summary...
§  New in Oracle Grid Infrastructure 12c:
§  Online Modification of Resources
LISTENERLISTENER
LSNR2NET1:1522LSNR2NET1:1522 LSNR2NET1:1528LSNR2NET1:1528
Adding a node listener to the second subnet
vixencomet
[GRID]> srvctl add listener -listener LSNR1NET2
-netnum 2 -endpoints 1522
[GRID]> srvctl config listener -listener LSNR1NET2
Name: LSNR1NET2
Home: <CRS home>
Listener LSNR1NET2 is not running
[GRID]> srvctl start listener -listener LSNR1NET2
Listener LSNR1NET2 is running on node(s):
§  Can I add another listener on another subnet?
§  Absolutely!
LSNR1NET2:1522LSNR1NET2:1522
Listener summary attached

9/30/13

27

Other listeners used as part of the stack
[GRID]> crsctl stat res –t
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
...
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE dasher STABLE
ora.MGMTLSNR
1 ONLINE ONLINE dasher 169.254.73.11 192.16
8.7.10 172.149.2.10,
STABLE
§  Oracle RAC 12c introduces more listener (types):
§  ora.LISTENER.lsnr
§  Default listener of type “ora.listener.type”
§  ora.ASMNETnLSNR_ASM.lsnr
§  An Oracle ASM listener used by Flex
ASM of type “ora.asm_listener.type”
§  ora.mgmtlsnr.type
§  Grid Infrastructure Management Repository
(GIMR) DB listener of type “ora.mgmtlsnr.type”
§  ora.LISTENER_SCANn.lsnr”
§  A SCAN listener of type “ora.scan_listener.type”
local resource (runs on every node)
cluster resource (runs on any node)
Valid Node Checking for Registration
§  New in the Oracle RAC 12c listeners:
§  For Oracle RAC: IPv6 support
§  Valid Node Checking for Registration
§  http://docs.oracle.com/cd/E16655_01/network.
121/e17610/listenercfg.htm#NETAG1588
§  The listener is now protected to both sides:
§  It can be restricted to accept client connections
§  http://docs.oracle.com/cd/E16655_01/network.
121/e17607/guidelines.htm#DBSEG504
§  It can be restricted to prevent ay arbitrary DB
(instance) from registering to it, if not allowed
by default. (Valid Node Checking for Registration)
vixencomet
LISTENERLISTENER
raccdb1_3raccdb1_1
Valid Node
Checking for
Registration
SCAN

9/30/13

28

SCAN in Oracle RAC 12c
The listener usage that combines it all
[GRID]> srvctl config network
Network 1 exists
Subnet IPv4: 10.1.1.0/255.255.255.0/eth0, static
Subnet IPv6:
[GRID]> srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
Registration invited nodes:
Registration invited subnets:
...
[GRID]> srvctl status scan_listener -netnum 2
SCAN Listener LISTENER_SCAN1_NET2 is enabled
SCAN listener LISTENER_SCAN1_NET2 is running on node vixen
§  SCAN was introduced with Oracle RAC 11g Rel. 2
–  http://www.oracle.com/technetwork/products/
clustering/overview/scan-129069.pdf
§  New in SCAN with Oracle RAC 12c:
–  Multiple Subnet Support
§  IPv6 support for client connectivity (public network)
–  Valid Node Checking for Registration
vixencomet
MySCAN_3 / 10.1.1.52
MySCAN2_3 / 10.2.2.52MySCAN2_2 / 10.2.2.51
Adding a second SCAN listener to the system
vixencomet
MySCAN / 10.1.1.50
§  Adding a second SCAN listener to the system
–  There is only one SCAN listener per subnet
–  In this example, each SCAN has only 1 IP address
§  Three steps are required to add another scan:
1.  Preparation of the host / network
2.  Adding another SCAN (VIPs mainly)
3.  Adding another SCAN listener
#Step 1: Preparation
[GRID]> more /etc/hosts
...
10.1.1.50 myscan myscan.localdomain
10.2.2.50 myscan2 myscan2.localdomain
[GRID]> srvctl config network
Network 1 exists
Subnet IPv4: 10.1.1.0/255.255.255.0/eth0, static
Subnet IPv6:
Network 2 exists
Subnet IPv4: 10.2.2.0/255.255.255.0/, static
Subnet IPv6:
The second (non-default)
network to be used for the
second scan (netnumber 2)
A second IP for SCAN2

9/30/13

29

Starting the SCAN listener on the second network
#Step 2: Adding another SCAN
[GRID]> su
Password:
[GRID]> srvctl add scan -scanname myscan2 -netnum 2
[GRID]> exit
[GRID]> srvctl add scan_listener -netnum 2 -endpoints 1530
[GRID]> srvctl status scan -netnum 2
SCAN VIP scan1_net2 is enabled
SCAN VIP scan1_net2 is not running
SCAN listener LISTENER_SCAN1_NET2 is not running
[GRID]> srvctl start scan_listener -netnum 2
SCAN listener LISTENER_SCAN1_NET2 is running on node vixen
vixencomet
MySCAN / 10.1.1.50
§  Then start the SCAN on the second network
LISTENER_SCAN1_NET2
Start using SCAN with the Oracle Database
[RAC]> sqlplus sys/<pswd>@MySCAN:1521/raccdb1 as sysdba
...
SQL> show parameters listener
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
listener_networks string
local_listener string (ADDRESS=(PROTOCOL=TCP)(HOST=
10.1.1.31)(PORT=1521)), (ADDRE
SS=(PROTOCOL=TCP)(HOST=10.1.1.
31)(PORT=1528))
remote_listener string myscan:1521
[RAC]> srvctl stop database -d raccdb1
[RAC]> srvctl start database -d raccdb1
[RAC]> sqlplus sys/oracle@MySCAN:1521/raccdb1 as sysdba
...
SQL> show parameters listener
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
listener_networks string (( NAME=net2)(LOCAL_LISTENER=(
DESCRIPTION=(ADDRESS=(PROTOCOL
=TCP)(HOST=10.2.2.32)(PORT=152
2))))), ((NAME=net2)(REMOTE_LI
STENER=myscan2:1530))
local_listener string (ADDRESS=(PROTOCOL=TCP)(HOST=
10.1.1.32)(PORT=1521)), (ADDRE
SS=(PROTOCOL=TCP)(HOST=10.1.1.
32)(PORT=1528))
remote_listener string myscan:1521
§  When using more than one SCAN on the system,
the LISTENER_NETWORKS parameter needs to
be set for databases.
§  New in Oracle RAC12c:
–  The LISTENER_NETWORKS parameter
is managed by the Oracle (Database) Agent
as part of the Oracle Clusterware management.
–  To initiate the new configuration
a database restart is required.

9/30/13

30

The SCAN listener FAQ
§  Can I define multiple ports? – YES**
§  Can you define another owner? – NO
§  Can you define another home? – NO
§  Can I add another listener on another port? – YES
§  See MOS Note 220970.1 – The RAC FAQ:
§  How to use SCAN and node
listeners with different ports?
• ** For the purpose of allowing applications to
maintain a distinguishable connect string on host /
port level using one SCAN as the host or to
enable port-based firewall configurations, SCAN
can be set up to support more than one port.
• It needs to be noticed that different ports are only
used for an initial connect.
• Once a connection using SCAN on any SCAN
port has been established, port assignment is not
maintained .
• This means that all SCAN ports are treated the
same, so that a connection that comes in on any
SCAN port can be directed to any registered local
listener on any port.
Agenda
–  Flex ASM
–  Flex Cluster

9/30/13

31

Graphic Section Divider

Understanding Oracle RAC 12c Internals OOW13 [CON8806]

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Understanding Oracle RAC 12c Internals OOW13 [CON8806]

Similar a Understanding Oracle RAC 12c Internals OOW13 [CON8806] (20)

Más de Markus Michalewicz

Más de Markus Michalewicz (20)

Último

Último (20)

Understanding Oracle RAC 12c Internals OOW13 [CON8806]