The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
MIgrating from Single Instance to RAC via Dataguard
1.
2. MIGRATING FROM SINGLE INSTANCE TO RAC AND
UPGRADING: A DATA GUARD SUCCESS STORY
Fuad Arshad @fuadar
Walgreen Company
3. Me, Myself & I
• More than 15 years of Oracle experience
• Worked in textiles, telecom & retail/healthcare
• Currently work for Walgreens in the Enterprise Shared
Services area
• Areas of Interest: backup & recovery, high availability,
MAA, performance tuning
5. Agenda
• Problem
• Analysis
• Decision
• Strategy
• Migration
• Things to look out for!
• Credits
6. Community Access
• RAC Attack Ask Questions
• IOUG RAC SIG
• Freelists Oracle L
• IOUG
• Stack Overflow
Blogs / Blogs /Blogs .
• Quora
• http://www.orana.info
• MOS Communities
• http://blogs.oracle.com
• Oracle Forums
• http://www.oaktable.net
7. The Problem
“It was very early, and we were still like beta or alpha
stage, and so we started receiving a ton of download.
The server became overloaded, and that's when I
realized that this had a huge market.”
Shawn Fanning (Napster)
Paper #371 7
8. Our Problem
• The application was growing and adding features
• Customer access slowdown was reported
Paper #371 8
9. Our Problem
• System built in the early 2000’s to handle the new
wave of technology
• More work as newer features were rolled out
• Architecture & design not intended to handle the
rate of change
• Database is always the bottleneck
Paper #371 9
10. Analysis
• Active/Passive servers
• Oracle database version close to desupport
• Old OS version
• Old SAN infrastructure
• CPU and disk I/O issues
Result -----> IT WAS OLD
Paper #371 10
11. Analysis
• Active/Passive Servers
• Oracle Database version close to desupport
• Old OS version
• Old SAN infrastructure
• Result ---------------- IT WAS OLD
Paper #371 11
13. Decision
• Let’s migrate to RAC
• Can it be done yesterday?
• Can we do it with zero downtime?
• The application team doesn’t want to change
anything
Paper #371 13
16. Pre-Migration
• The strategy was to build a RAC database from a single
instance environment
• Oracle provided instructions on how to prep the single instance
environment so that it is RAC-aware. This includes running
catclust scripts and adding undo tablespaces for the number of
nodes that the new cluster will have
Paper #371 16
17. Pre-Migration
• The server builds need to have all the patches based on the OS
that is being used
• Oracle provides a variety of starter kits that can be used to
validate the configuration
• It is also important that the network be configured and available
based on best practices
• 11.2.0.2 and above have added requirements like SCAN that
need to be accounted for
Paper #371 17
18. Pre-Migration
• RAC Check is important tool to ensure that the environment is
built based on best practices
• RAC Check can be downloaded at MOS # 1268927.1 and is
updated frequently
– OS settings including kernel parameters and patches
– CRS/Grid Infrastructure
– RDBMS memory and database parameters
– ASM parameters and statistics
Paper #371 18
19. Pre-Migration (Primary)
Data Guard setup steps need to be followed which include :
• Enable supplemental logging & force logging on primary
• Add additional REDO threads (Based on # of RAC nodes)
• Add additional UNDO tablespaces (Based on # of RAC nodes)
• Backup the primary database
• Create a PFILE from the primary database
• Copy TNSNAMES and LISTENER files into a staging directory
• Configure the primary database for Data Guard
Paper #371 19
20. Pre-Migration
• RAC Check is important tool to ensure that the environment is
built based on best practices
• RAC Check can be downloaded at MOS # 1268927.1 and is
updated frequently
– OS settings including kernel parameters and patches
– CRS/Grid Infrastructure
– RDBMS memory and database parameters
– ASM parameters and statistics
Paper #371 20
21. Pre-Migration (Standby RAC)
• Data Guard setup steps need to be followed which include:
• NFS mount or copy backup and auxiliary files
• Configure Oracle Net which includes setting up TNSNAMES and LISTENER
for 11.2.0.2 (This will be done in the Grid infrastructure home)
• Modify the TNSNAMES.ORA to include the primary as well as the standby
tns entries
• Create a tns entry for the listener and make sure it is referenced in the
REMOTE_LISTENER (REMOTE_LISTENER specifies the global list of
listeners in a RAC configuration)
• Make sure the listener alias referenced by the LOCAL_LISTENER
initialization parameter is in each standby tnsnames.ora file
• Ensure that the standby server has the same software version and patches
installed as the primary Paper #371 21
22. Pre-Migration (Standby RAC, cont.)
Data Guard setup steps need to be followed which include:
• Ensure that a password file is configured on both the primary and the
standby
• Copy the PFILE to the $ORACLE_HOME/dbs in the standby node
• Add DB_UNIQUE_NAME and CLUSTER_DATABASE parameters in the
PFILE
• Modify all the other parameters outlined in the white paper
• Connect to the standby and create an SPFILE in the ASM diskgroup
• Startup the proposed standby environment in NOMOUNT mode
• Duplicate the database using “DUPLICATE TARGET DATABASE FOR
STANDBY”
• Setup Server Control (SRVCTL) to register the instances
Paper #371 22
23. Pre-Migration (Standby RAC, cont.)
Data Guard setup steps need to be followed which include:
• Add additional standby REDO logs (The number of standby redo logs should
be one more than the number of online REDO logs per thread and should be
the same size)
• Enable Managed Recovery Mode on the standby database
• Enable archive destination on the primary to start shipping logs to the
standby
• Validate the transfer
• Setup Data Guard Broker
• Validate the configuration and ensure that logs are being applied
• Bring up the other nodes in the RAC environment
Paper #371 23
24. Migration
Migrating is simple after the Data Guard database is setup
• Ensure all ARCHIVELOGS have been applied
• Put the primary database in restricted mode and switch ARCHIVELOGS
• After all the outstanding logs have been applied, shutdown REDO apply on
the standby
• Validate SCN across the primary and the standby
• Two options from here:
- Switchover & upgrade
- Shutdown & upgrade
**We Chose Shutdown & Upgrade**
Paper #371 24
25. Migration
• All standby RAC nodes can be up if using DBUA
• Ensure pre-checks are done
• DBUA is brutal if you don’t remove the obsolete parameters
• DST changes are important
• If manually upgrading DBMS_DST needs to be run to upgrade DST
Paper #371 25
26. Things to Look Out For
Ensure Environments Are Validated
• RDA should be run and all issues fixed
• RAC check will give you a detailed overview of the system
• Ensure all starter kits and the RAC manuals are consulted: the result can be
a good best practices evolving document
• Use services to better manage how the application connects and load
balance your system
• Private network is key with RAC: follow Oracle’s best practices.
• XA changes in 11.2.0.2 & can cause issues: important to understand them
• “Compatible” parameter testing is important.
• Monitoring the environment after go-live is very important
Paper #371 26
27. Credits
I’d like to thank the following people for helping me with the
presentation and accompanying white paper:
• Matthew Gallagher, Architect -- Walgreens
• Brice Lahl, Manager DB Architecture -- Walgreens
• Daniel Morgan, ACE Director -- Drugstore.com
A special shout out to Arup Nanda & Alex Gorbachev for encouraging me to present and
Leighton Smith for reviewing my session
Also special credit to the RAC Assurance Team at Oracle for helping is get through this migration and
subsequent issues. Some content in the whitepaper and presentation has been republished with permission from
the Oracle RAC Assurance Team whose goal had been to proactively provide input to customers with new and
existing CRS, RAC, ASM and MAA implementations.
Paper #371 27
While the chart shows about an average CPU consumption. What is doesn’t show it the what you see in the High Value Seasonal Adjustment which was 92 %. Now we Always had and Still have What Tom /Graham and Andrew talk about too many connections ratio to the the cores available
This was a plan presented to upgrade the hardware in current form.
Decisions are always hard and are always made late. The users expect that time to market after a decision to be immediate.
Adding the dg_config,archive_dest fal_server and fal_client can be done ahead of time with the log_archive_dest_state be disabled till the standby instance is up
It is very important to understand that the infrastructure has to be ready and support a RAC environment. The SPFILE shouldb e on a shared Accessible point as well as all nodes need to be in the GI already
It is very important to ensure that fal_client and fal_server are set up correctly. Dataguard Broker can be setup with a few caveats since it is not officially supported on an Active/Passive Cluster. The Solution to Having a Active/Passive Cluster Working with a Broker is setting the local_listener parameter to the Virtual NodeName/IP
Prechecks like utl112i not being run and validated can cause issue. We have seen cases where if too many versions are in between the upgrades utl112 might not catch an obsolete parameter and cause issues with DBUA. We tested the process with manual upgrades and DBUA. DBUA is much cleaner in newer versions and allows for getting everything right. Logging is better.
Prechecks like utl112i not being run and validated can cause issue. We have seen cases where if too many versions are in between the upgrades utl112 might not catch an obsolete parameter and cause issues with DBUA. We tested the process with manual upgrades and DBUA. DBUA is much cleaner in newer versions and allows for getting everything right. Logging is better.
RDA can show various errors as as allow for a DBA to health check their environment. Raccheck is a health Audit Tool. Starter Kits are important.Services can help your application. Look and understand before impl