SlideShare una empresa de Scribd logo
1 de 78
1
<Insert Picture Here>
Tuning All Layers Of E-Business Suite – Performance Topics
Isam Alyousfi, Senior Director
Lester Gutierrez, Senior Architect, EBS Applications Performance
3
The following is intended to outline our general product
direction. It is intended for information purposes only,
and may not be incorporated into any contract. It is
not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making
purchasing decisions.
The development, release, and timing of any features
or functionality described for Oracle’s products remains
at the sole discretion of Oracle.
4
5
Objectives
• Overview of Performance Methodology for Optimizing E-
business suite
• Outline best practices for Tuning E-Business layers
– Tuning the Applications Tier
– Tuning the Concurrent Manager
– Tuning the Database Tier
– Tuning the Applications
6
R12 Apps Basic Layers / Products
B
r
o
w
s
er
Lis
ten
er
mod_oc4j
mod_proxy
mod_plsql
mod_osso
Apache
JVM1 App1 App2
OC4
J
oc4j_security
LDAP
Oracle Forms
Repository
OID
RAC Database
+ db Listener
IdM
JVM2 App3 App2
Concurrent
Processor,
Manager
Ext.
procs
OAM
7
R12 Apps Basic Layers / Products
8
Application & SQL
Management
System Resource
Management
Space
Management
Backup & Recovery
Management
Storage
Management
Database
Management
Procedural stmts. Procedural
Statement
ExecutorSQL stmts.
SQL Statement Executor
PL/SQL
Block
PL/SQL engine
SQL engine
CPU
SGA
Memory
SQLNET
JDK
OC4J
JAVA
Forms
JDBC
OAFramework
XML
BI Publisher
Buffer Cache
WF
Concurrency
Discoverer
Optimizer
Access Path
Indexes
600,000+ SQL statements
200 + Products
FND, GL, AP, AR, OM, SC, …
Concurrent manager
MLS
100,000 + PLSQL packages
40,000 + Tables
OJSP
OCI
SOA
BPEL
Out of Memory
99
Performance Methodology for Optimizing of E-business suite
• How to Approach a Performance Issue
• Where is the time going ? Get the right diagnostics
– Where?
– What?
– Why?
1010
How to Approach a Performance Issue
1. DEFINE the problem clearly
1. GATHER the right data to analyze the issue
1. Identify the ROOT CAUSE of the problem, possibly gather additional data
1. Search for a KNOWN SOLUTION or workaround that addresses the root
cause of the problem
1. If it is a product issue, PASS ON the right information to support/dev through
the regular channels
1. Try to identify a TEMPORARY WORKAROUND to alleviate the issue while
you get a product fix
1111
Define the problem
• Get a clear understanding of the performance issue
What
Where
When
Why
How
1212
Define the problem
Sequence Diagram
1313
Gather data
• Start with Best Practices : (note:
1121043.1)
• SQL Tuning
– Trace files
– SQLT output (note: 215187.1)
– Trace Analyzer (note: 224270.1)
– AWR Report (note: 748642.1)
– AWR SQL Report (awrsqrpt.sql)
– 11g SQL Monitoring
– SQL Tuning Advisor
• PL/SQL Tuning
– Product logs
– PL/SQL Profiler (note: 808005.1)
• Reports Tracing
• note: 111311.1
•D
atabase Tuning
• AWR Report (note: 748642.1)
• ADDM report (note: 250655.1)
• Automated Session History (ASH)
Report
• LTOM output (note: 352363.1)
•F
orms Tuning
• Forms Tracing (note: 373548.1)
• FRD Log (note: 445166.1)
• Generic note: 438652.1
•M
iddletier Tuning
• JVM Logs
• JVM Sizing (note: 362851.1)
• JDBC Tuning (note: 278868.1)
•O
14
Root Cause Analysis
• Where is the time going?
• Who is consuming time?
• Why?
15
Where is the time going ? Get the right diagnostics
• Identify the components that do the processing
– Refer to the architecture
• which tier(s) play a part ?
• which components within a tier come into play ?
• 80% of issues are DB related
16
17
Root Cause Analysis: Check the basics
• Version levels
• Required Configuration
– Init.ora : Notes 396009.1 (R12) & 216205.1 (11i)
• Latest patch levels
• Recommended Patch lists
– Note “Recommended Performance Patches for the Oracle E-Business Suite”, 244040.1
– Also check for product specific notes
• Available Fixes
• Best Practices in Performance for EBS
– http://blogs.oracle.com/stevenChan/
18
Where is the time going ? Get the right diagnostics
• If non-database (client side) processing occurs within the flow
that was traced, you may see wait events like
– SQL*Net Message%
– TCP Socket%
– PLSQL Lock Timer
• Other useful data in identifying where time is spent
– Raw sql trace file
– Product logs
– App server logs such as http logs
– Client process cpu consumption from OS (for example ps)
19
Tuning the Application Tier
20
Tuning the Applications Tier
•Forms
–To reduce load on the database server, ensure users are optimally
utilizing the professional Forms interfaces.
–Avoid Blind queries
–Provide selective criteria in Find windows and LOVs
–Minimize network traffic and form open times
–Avoid opening and closing forms across transactions
–Combine forms from multiple-products onto a single menu
–Generate complete SQL trace and Forms Runtime Diagnostics
(FRD) to debug performance issues
2121
Where is the time going ? Get the right diagnostics
• Client Process CPU Consumption
– Help -> Oracle Applications
– Forms and Database process id & CPU Time
22
Tuning the Applications Tier
OC4J/JVM
•Use one JVM per 2 CPUs
•~100 concurrent users per JVM
23
Tuning the Applications Tier
OC4J
•Response Time/CPU Usage
•OutOfMemoryErrors
24
Tuning the Applications Tier
•First, rule out any SQL issues.
–Note 357597.1 on how to enable SQL trace for OA Framework
applications
•If there are no database-related issues, then
–you need to analyze the JVM
•Techniques you can use:
–Thread dumps – Send to Oracle Support for analysis
–GC logs: Configure JVM sizing
–Profilers - Send to Oracle Support for analysis
Response Time/CPU Issues
25
Tuning the Applications Tier
•GC tuning
–Total available memory is the most important factor affecting GC
performance.
–Review the frequency of collections, especially major collections
(i.e. Full GC)
–Enable verbose GC to tune heap sizes based on the GC traffic
–Start with: -Xms1024M and –Xmx1024M
–Setting -Xms and -Xmx to the same value avoid the need to memory
allocation during runtime
–If full GCs are too frequent, consider increasing Xms and Xmx
26
Tuning the Applications Tier
•Common causes of OutOfMemoryError
–Memory Leak – Some objects not cleaned up
•Strong reached references, connection objects, etc.
–Memory Hemorrhage –
•Some kind of infinite loop
•Reading large data sets consuming vast amounts of memory in a short
period of time
•Connection leaks leading to vast object allocations
•SQL Statements/caching may lead to excess memory used in JDBC
libraries
–Sizing Issues – Heap is undersized for expected load
27
Tuning the Applications Tier
•Symptoms of memory leak
–Heap usage increases steadily, over a relative longer period of
time (e.g. serveral hours or days)
–When the amount of memory leak is large enough, you will start
seeing continuous Full GC in the GC log
–Different users may encounter OOM on totally unrelated flows
–Generate Heap Dump files when OutOfMemoryError occurs
•Help Oracle Support identify the root cause of the leaks
28
OA Fwk : JVM Process ID
Process ID of the JVM handling your session
•
Client Process CPU Consumption
• OA FWK – JVM Process ID
• The “About” link is at the bottom of each page
• Requires FND_DIAGNOSTICS profile set to
29
<Insert Picture Here>
Tuning the Concurrent Manager
30
Tuning the Concurrent Manager
–To maximize throughput for jobs which spawn parallel workers (i.e. Auto
Invoice, Payroll), consider reducing the sleep time of the Conflict Resolution
Manager (CRM)
• Default is 60s, consider 5 or 10 seconds
–Increase the cache size (number of requests cached) to at least twice the
number of target processes
–If the CM and Database are co-located, use IPC or bequeath connection
–Consider dedicating certain concurrent managers to process either short or
long running programs to avoid queue backup
–Purge the FND tables on a regular basis using the “Purge Concurrent
Request and Manager Data” program
31
Tuning the Concurrent Manager
•Workload Management
–Trim the fat. Review and eliminate concurrent jobs that are not
required and/or are not being used by your users.
–Avoid enabling an excessive number of standard or specialized
managers
–A common guideline is between 1-2 target process per CPU, but
this needs to be balanced with resources required by online
activity
32
Tuning the Concurrent Manager
•Workload Management
–Use specialization rules and work shifts to bind specific jobs to
specific time windows
•Helps avoid scheduling resource intensive batch requests during
peak activity
–Reschedule some programs to run when the concurrent managers
have excess capacity and
–Add more queues if all queues are running at maximum capacity
only if resources permit.
33
Tuning the Concurrent Manager
•Transaction Managers - TMs
–Used for synchronous online processing(ex:Inventory Transactions)
–Ensure enough TMs exist to service the request load
•Set the profile “Concurrent:Wait for Available TM” to 1 (second).
–Set the sleep time on the TMs to a high number (e.g. 10 minutes)
•Avoids constant polls to check for shutdown requests
34
<Insert Picture Here>
Tuning the Client & Network
35
Tuning the Client
–Minimize the browser memory footprint
•Minimize toolbars, browser themes, extensions, plug-ins, helper applications,
and phishing controls
•Do not minimize E-Business Suite components (but minimize other apps)
•If there is a choice, use OAF/HTML (smaller memory profile)
–Upgrading RAM is generally more important than CPU
–Always keep menus small (customize if needed)
•Menus are both memory and network intensive
–Oracle E-Business Suite 11i and 12 PC Client Performance
•http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html
36
Tuning the Network
–Routing Issues
–Lost Packets due to device saturation
–DB and MT must be co-located on a local
private network
–Traffic from MT to Client:
•Consider using Local proxy in local offices (global single instance), (note
153287.1)
37
Tuning the Database
38
Tuning the Database Tier – Configuration
• Ensure mandatory init.ora parameters are set correctly
– MOS Docs 396009.1 & 216205.1 (R12 & 11i respectively)
– Use AWR advisories to optimize SGA/PGA sizing
• Required Patches & Fixes/Workarounds for Known Issues
– Recommended Performance Patches -MOS Doc 244040.1
– Required Patching – Interoperability Notes
•11gR1 Certified with 11i and R12 – MOS Docs 452783.1 & 735276.1
•11gR2 Certified with 11i – MOS Doc 881505.1
•10gR2 Uptake patch bundle 8362683, MOS Doc 791049.1
39
Tuning the Database Tier – Configuration
• Use key Database features
– Auto Memory Management
• Exception – Linux HugePages – MOS Doc 744769.1
– System Managed Undo
– Auto Segment Space Management
– PL/SQL Native Compilation
• Convert to the OATM Tablespace Model
– Consolidates # of Apps tablespaces from 400 to ~12
• MOS Doc 248857.1
– Best practices: LMT, ASSM, groups objects by size/type/access pattern
– Migration utility can be used to migrate a schema at a time
– Use AUTOALLOCATE extent sizes
40
Tuning the Database Tier – Disk IO
• I/O Optimization
– Configure & verify direct IO is used if using file system datafiles
• Improves performance/scalability by simulating raw devices, avoiding
file system cache traffic
– Ensure your IO subsystem can handle your peak IO load & IOPS
• RAID, S.A.M.E. configs with sufficient spindles
• Measure with ORION tool or in 11g
DBMS_RESOURCE_MANAGER.CALIBRATE_IO – MOS Doc 727062.1
– Reduce the IO workload
• Tune expensive SQL
• Trim the workload of unnecessary tasks
• Do 3X less IO on average with 11g Advanced Compression Option
41
Tuning the Database Tier – Statistics Gathering
• Use only FND_STATS or the Gather Schema / Table Statistics Concurrent
Programs
– Do NOT USE the analyze or dbms_stats command directly. It is not supported, and
can result in sub-optimal plans
• Use the GATHER_AUTO option to gather incrementally
• Analyze all schemas at 10%, then specific objects at x%
– Due to data skew, some tables benefit from higher sampling %
– For 11g, can use value of zero (AUTO_SAMPLE_SIZE)
• Disable the 10g/11g automatic job to gather stats
42
Tuning the Database Tier – Statistics Gathering
• Consider locking stats for very volatile tables once a representative set
is gathered
– Some Interface tables
– Other examples: ap_selected_invoices, wsh_pr_workers
– Use FND_STATS.LOAD_XCLUD_TAB to skip the table
• Do not gather statistics excessively on entire schemas or the entire
database such as nightly or weekly
• FND_STATS now supports Database 11g Extended Stats. Patch
9470196. Also, in patch set 9962815
43
Tuning the Database Tier – Performance Management
• Establish baselines for different workloads
–Use AWR baselines & diff reports
•DBMS_WORKLOAD_REPOSITORY or Enterprise Manager
•OS metrics with tools such as OS Watcher – MOS Doc 301137.1
–Timings/Traces for critical flows & concurrent programs
• Use 11g’s SQL Performance Analyzer (SPA) & SQL Plan
Management features (SPM)
–Change management performance impact
–Stabilize execution plans as needed & quickly restore “good” plans while
analysis and permanent solutions are found
44
Tuning the Database Tier - AWR
• Regularly Review Automatic Workload Repository data
• Review Top 5 Timed Events section to determine utilization &
bottlenecks and review relevant detailed sections for these
45
Tuning the Database Tier - AWR
• For latch related waits
– Often due to non-sharable SQL, sub-optimal SQL which performs full table
or full index scans, dynamic object creation/removal, etc.
– Review the latch Statistics section to determine the hot latches
– Trace some waiter and holder sessions to determine actual cause
• For I/O related waits
– Review SQL sections by Logical/Physical reads and the Segment Statistics
sections by I/O
• For enqueue related or buffer busy waits
– Review the following sections: enqueue, segment statistics by buffer busy
waits, row lock & ITL waits
46
Tuning the Database Tier - AWR
• Link AWR with ASH data
– Map sessions & SQL to main bottlenecks
– Query Active Session History (ASH) data
– Enable tracing for a few of the sessions with heavy waits determine the actual
cause and SQL statements.
– Review the Advisory sections in AWR to fine tune SGA & PGA
– Correlate top SQL in AWR to business flows
– Produce and review SQL AWR reports for top SQLs (awrsqrpt.sql)
– Monitor OS Stats (included in 11g AWR)
• CPU and Memory Utilization & Paging, I/O statistics & response times
47
Tuning the Database Tier – DB Console
48
Tuning the Database Tier – DB Console
49
• If bottleneck is IO related...
(db file sequential read, db file scattered read, log file sync, free buffer waits, etc...)
– Tune Top SQL is always the first step
– Maximize Memory availability, allocate generously to buffer cache, review
AWR advisories & monitor swapping & paging
– Use ASM or alternatively use the SAME methodology for db files
– IO Sub-system:
• RAID 10 still preferred most often for high end requirements Ideal avg.
response times of < 10ms.
– Check for excessive/redundant indexing
– Configure async IO, use quick IO technologies
Tuning the Database Tier –
Common corrective actions to maximize scalability
50
• If the bottleneck is concurrency related (enq
%, latch%, buffer busy%, etc )
– Use global hash-partitioned indexes for hot leaf blocks - Identify via AWR "Top
logical IOs by Segment"
– Some of these waits are commonly caused by bad SQL execution plans
– i.e. :latch: cache buffer chains, %buffer busy waits, read by other session
– Make sure to use ASSM and OATM tablespace model.
– Increase INITRANS to alleviate ITL contention
– Increase sequence caches
– Work with support/consulting to evaluate if table/index partitioning will help
– If the program allows, more frequent commits (batch size) can help by reducing the
CR (consistent read) work required
Tuning the Database Tier –
Common corrective actions to maximize scalability
51
• If the bottleneck is RAC related ... (gc
%, global%, etc...)
– First do all the steps mentioned previously...
– Check interconnect configuration -- Use Jumbo frames.
– Do not use parallel query/dml across nodes
• Use instance_groups & parallel_instance_groups settings
– Review Parallel Concurrent Processing & Application Affinity config
• 12.1.3 introduced option to define node affinity at the program level –
Concurrent -> Program -> Define
– Perform RAC focused AWR analysis …
Tuning the Database Tier –
Common corrective actions to maximize scalability
52
Tuning the Database Tier –
Common corrective actions to maximize scalability
• RAC AWR Analysis
1.Verify SQL execution is optimized – 10046 trace/SQL Monitoring
2.Collect AWR data from all instances
- awrgrpt.sql and awrgdrpt.sql (cluster-wide base and diff reports)
1.Verify IO, CPU utilization and Interconnect messaging performance are within
acceptable limits
2.Drill down on cluster wait events + “SQL order by Cluster Waits”
3.Correlate SQL executions on all instances accessing/modifying top segments in
Segment Statistics by
• Global Cache Buffer Busy & Buffer Busy Waits
• CR/Current Blocks Received
• Logical Reads
53
• To get full value from tracing…
– Raw trace must be
• A complete, non-truncated trace file
*** DUMP FILE SIZE IS LIMITED TO 12345 BYTES***
• From before the user action began To the point the DB session ends**
– Tkprof output MUST have
• Runtime Execution Plans and Row Counts & Row Source Stats
• Run Time Execution Plan = “Row Source Operation” != “Explain Plan”
– Best practices, detailed how-to steps & recommended tracing patches documented
• MOS Doc 1121043.1
-Also review Appendix B – Getting In-Memory partial runtime stats
Tuning the Database Tier – TKPROF: Good, Bad & Ugly
54
Tuning the Database Tier – Good TKPROF
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 7674758 780.79 728.70 0 1 0 0
Fetch 7674758 6328.78 6757.80 225987 153846532 0 7674758
------- -------- -------- ---------- ---------- ---------- ---------- ----------
total 15349517 7009.58 7486.52 225987 153846533 0 7674758
Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 173 (recursive depth: 2)
Rows Row Source Operation
------- ---------------------------------------------------
7674758 SORT AGGREGATE (cr=153846532 pr=225987 pw=0 time=7174772779 us)
7675089 NESTED LOOPS (cr=153846532 pr=225987 pw=0 time=6919378560 us)
7675089 SORT UNIQUE (cr=123146176 pr=162809 pw=0 time=6083080435 us)
55878643 TABLE ACCESS BY INDEX ROWID AR_TRX_COGS_GT (cr=123146176 pr=162809 pw=0 time=5577872837 us)
136390387 INDEX RANGE SCAN AR_TRX_COGS_N1 (cr=23439707 pr=37706 pw=0 time=2136416980 us)(object id..
7675089 TABLE ACCESS BY INDEX ROWID RA_CUSTOMER_TRX_LINES_ALL (cr=30700356 pr=63178 pw=0 time=539908369 us)
7675089 INDEX UNIQUE SCAN RA_CUSTOMER_TRX_LINES_U1 (cr=23025267 pr=2820 pw=0 time=196155435 us)(object ..
55
Tuning the Database Tier
• Why upgrade EBS to
– Internal tests show 10-20% improvement in EBS batch & reporting workloads
– Significant RAC Performance improvements
– Oracle internal’s production EBS system on 11gR1;
•11gR2 scheduled for the summer
– Lower cost with Advanced compression, better/automated diagnostics,
lower-risk upgrade with SPM/RAT/SPA …
– Best Practices/Tips white papers
•twp_upgrading_10g_to_11g_what_to_expect_from_optimizer.pdf
•http://www.oracle.com/apps_benchmark
56
11g SQL Tracing Enhancements
– New tracing parameter PLAN_STAT
• first_execution | all_executions | adaptive*
– Connection pool problem to get row source stats is solved
– Key tracing patches* listed on MOS Doc 1121043.1
– Turn trace on for just 1 or a list of sql_id’s
– With real-time row source stats monitoring from
v$sql_plan_statistics – no more waiting hours for a complete trace to
get the right trace data
– Expanded event syntax :
- alter session set events ’sql_trace [sql:a5ks9fhw2v9s1|56bs32ukywdsq] …
- sql_trace wait=true | false, bind=true | false,planstat=never | first_execution |
all_executionss|adaptive level = 1| 4| 8| 12 | 16
57
11g Automatic SQL Monitoring & Tuning
58
11g Performance Enhancements
• Optimizer improvements
– SPM (SQL Plan Management)– quick relief while permanent solution arrives
– Better density estimates for skewed data
– Faster, more accurate statistics with AUTO_SAMPLE_SIZE
– Extended statistics - address predicate correlation
– Adaptive cursor sharing – to address bind peeking issues
• Some others…
– Interval partitioning, Additional partitioning schemes
– Text indexing performance enhancements
– RAC scales better & AWR is more RAC aware
– Many others..
59
11g Advanced Compression
• Data compression for all data types
• Structured, Unstructured, Backup, Network
• Reduces storage consumption by
2 to 4 times
• Improves read performance
• Enhances memory, buffer cache
utilization
• Complete application transparency
• Benefits diverse application
workloads
Initially
Uncompressed
Block
Compressed
Block
Partially
Compressed
Block
Compressed
Block
Empty
Block
60
• Oracle Database 11g extends table compression for
OLTP data
– Support for conventional DML Operations (INSERT, UPDATE)
– New algorithm significantly reduces write overhead
• Batched compression minimizes impact on OLTP transactions
– No impact on reads
• Reads data directly in its compressed format
• Many reads actually see improved performance due to fewer I/Os
and enhanced memory efficiency
11g Advanced Compression
61
Leaf Blocks
Nancy,ROWID
Nancy,ROWID
Nancy,ROWID
Nicole,ROWID
Norm,ROWID
Pablo,ROWID
Paula,ROWID
Paula,ROWID
Peter,ROWID
Luis,ROWID
Mark,ROWID
Mary,ROWID
Mike,ROWID
Mike,ROWID
Karl,ROWID
Kathy,ROWID
Kim,ROWID
Lance,ROWID
Pierre,ROWID
Rachel,ROWID
Rajiv,ROWID
Raoul,ROWID
Phil,ROWID
F
H
Kar
N
P
Ph
B
C
Cr
Sam
St
Su
Di Lu Rh
11g Advanced Compression – Index Compression
• Compress multi-column indexes to
reduce the storage overhead of
repeated values
• Stores more keys per leaf block
• Less I/O,better performance of
range scans
62
11g Advanced Compression - Compression Advisor
• Estimate Storage Savings for your case
– Available in 11g Release 2 or …
– Available on OTN * for previous releases
– Shows projected compression ratio for uncompressed tables
– Reports actual compression ratio for compressed tables (11g Only)
* http://www.oracle.com/technology/products/database/compression/compression-advisor.html
63
•Oracle’s Internal E-Business Suite Production system deployed ACO in ‘09
–Average overall storage savings - 3x
•Table compression 4x
•Index compression 2x
•LOB compression 2.3x
–231TB of realized storage savings primary, standby & test systems
•Additional benefits also accrued in Dev clones and backup
•Payroll, Order-2-Cash, AP/AR batch flows, Self-Service flows run without
regression, Queries involving full table scans show speedup
11g Advanced Compression with E-Business Suite
64
• White paper: Upgrade to Oracle 11g - Performance Best Practices
http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html
• Special attention to
– Testing of 11g or 10g workloads with Real Application Testing
– Managing SQL Execution Plans and minimizing performance
degradation with11g SQL Plan Management
11g Upgrade Best Practices for EBS
65
Tuning the Applications
66
Tuning the Applications
• MyOracleSuport note “Recommended Performance
Patches for the Oracle E-Business Suite”, 244040.1
– Recommended performance patches for all the modules and
tech. stack components are consolidated in this note
• Apply latest Release Update Packs
• Functionality, fixes, and updates for all Release 12 functional
applications and the technology stack
• Several product family RUPs included are also available
individually
67
Tuning the Applications
• Apply latest Release Update Packs
– Release 12.1.3: Patch 9239090
• MOS Doc: 1080973.1
– Release 12.1 : Patch 7303030
• MOS Doc: 752619.1
– Release 12.0 - RUP6: Patch 6728000
• MOS Doc: 743368.1
• Review EBS Benchmark Publications
• http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html
68
Tuning the Applications
• Performance Best Practices
– Workflow
• For background engines via Concurrent Manager, set the ‘Process Stuck’
parameter to ‘No’
– Start a separate background engine to handle stuck processes with a low
frequency –i.e. once or twice a day
• Use deferred activities to improve online response times for flows such as
Scheduling, PO Document Approval, etc.
• Frequently purge runtime data (Conc Program: FNDWFPR)
• Run $FND_TOP/sql/wffngen.sql to translate the activity function calls into
static calls
• Disable retention on Workflow Queues
– DBMS_AQADM.ALTER_QUEUE(queue_name=>:b1,retention_time=>0);
• For high volume batch processing in RAC use ITEM_TYPE to Node affinity
69
Tuning the Applications
• Performance Best Practices
– Purge & Archive
• Implement a regular purge process including transient data as
well as obsolete transactional data
• Purge old data before performing upgrades
• Many documented standard programs available
– ~260 purge programs in R12
• Purge Portal
– Purge programs can be configured, initiated and monitored
• Set the execution frequency as well view history of purge
programs
– R12: programs tagged with the “Purge” program type
70
• Runtime Performance Testing Tips
– Use Automated, scripted tools
• EBS Test Started Kits (Winrunner/QTP)
– Bundled QA based automated scripts for EBS testing - Patch 8408886
• Oracle Applications Testing Suite (Accelerators for EBS)
– Web and Forms based flows
– Complement with user participation tests and batch load tests with frequent
and critical jobs
– References
http://blogs.oracle.com/stevenChan/2009/10/oats_ebs_certified.html
http://blogs.oracle.com/stevenChan/2009/08/evolutionary_steps_automated_testing_ebs.html
Tuning the Applications
71
References
72
References
• R12.1 documentation roadmap (790942.1)
• “Oracle E-Business Suite Release 12.1 Info center” (806593.1)
• Database preparation guidelines for R12.1 upgrade (761570.1)
• Patching FAQs (459156.1, 225165.1)
• Using staged or shared APPL_TOP and distributed AD (734025.1,
384248.1, 236469.1)
• OAM “Patch Wizard” overview and FAQ (976188.1, 976688.1)
• AD Command Line Options for Release R12 (1078973.1)
• EBS 12.1.3 Data Model Comparison Report (1290886.1)
• Recommended Performance Fixes (244040.1)
• R12 Upgrade Sizing & Best Practices (399362.1)
73
SUMMARY
DO’s..
•Read and follow official documentation and
have a project plan
•Engage Support early on to validate your
upgrade plan
•Identify and execute tasks you can do today
•Test ! Test ! Test !
•Keep track of patches applied in test upgrades
•Be smart about using the right tools and
explore published downtime reduction
techniques
•Optimize patch runs to suite your H/W
DON’Ts..
•Ignore including relevant functional SMEs in
the planning process from the beginning
•Skip failed or long running jobs as a regular
practice
•Drop indexes
•Run adpatch in serial mode
•Perform workarounds without approval from
support/DEV
•Insufficient time and/or # of rounds of upgrade
and critical functional flow testing
74
Oracle Products Available Online
Oracle Store
Buy Oracle license and support online
today at
oracle.com/store
75
76
77
A
Q&
78

Más contenido relacionado

La actualidad más candente

New Approaches to Faster Oracle Forms System Performance
New Approaches to Faster Oracle Forms System PerformanceNew Approaches to Faster Oracle Forms System Performance
New Approaches to Faster Oracle Forms System PerformanceCorrelsense
 
Database Performance Tuning Introduction
Database  Performance Tuning IntroductionDatabase  Performance Tuning Introduction
Database Performance Tuning IntroductionMyOnlineITCourses
 
Sql server 2008r2 dba course
Sql server 2008r2 dba courseSql server 2008r2 dba course
Sql server 2008r2 dba coursesssql
 
Analyzing OTM Logs and Troubleshooting
Analyzing OTM Logs and TroubleshootingAnalyzing OTM Logs and Troubleshooting
Analyzing OTM Logs and TroubleshootingMavenWire
 
DB12c: All You Need to Know About the Resource Manager
DB12c: All You Need to Know About the Resource ManagerDB12c: All You Need to Know About the Resource Manager
DB12c: All You Need to Know About the Resource ManagerAndrejs Vorobjovs
 
Oracle applications r12.2, ebr, online patching means lot of work for devel...
Oracle applications r12.2, ebr, online patching   means lot of work for devel...Oracle applications r12.2, ebr, online patching   means lot of work for devel...
Oracle applications r12.2, ebr, online patching means lot of work for devel...Ajith Narayanan
 
How to Turn New Recruits Into Oracle EPM Infrastructure Gurus
How to Turn New Recruits Into Oracle EPM Infrastructure GurusHow to Turn New Recruits Into Oracle EPM Infrastructure Gurus
How to Turn New Recruits Into Oracle EPM Infrastructure Gurusnking821
 
Ten query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowTen query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowKevin Kline
 
Sql server 2012 dba online training
Sql server 2012 dba online trainingSql server 2012 dba online training
Sql server 2012 dba online trainingsssql
 
Oracle performance tuning_sfsf
Oracle performance tuning_sfsfOracle performance tuning_sfsf
Oracle performance tuning_sfsfMao Geng
 
EM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsEM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsMaaz Anjum
 
SQL Server Tuning to Improve Database Performance
SQL Server Tuning to Improve Database PerformanceSQL Server Tuning to Improve Database Performance
SQL Server Tuning to Improve Database PerformanceMark Ginnebaugh
 
Windows 7 For Geeks
Windows 7 For GeeksWindows 7 For Geeks
Windows 7 For GeeksAdil Mughal
 
Oracle Enterprise Manager 12c - OEM12c Presentation
Oracle Enterprise Manager 12c - OEM12c PresentationOracle Enterprise Manager 12c - OEM12c Presentation
Oracle Enterprise Manager 12c - OEM12c PresentationFrancisco Alvarez
 
Getting optimal performance from oracle e-business suite presentation
Getting optimal performance from oracle e-business suite presentationGetting optimal performance from oracle e-business suite presentation
Getting optimal performance from oracle e-business suite presentationBerry Clemens
 
Nicholas king oracle epm migration and upgrade
Nicholas king   oracle epm migration and upgradeNicholas king   oracle epm migration and upgrade
Nicholas king oracle epm migration and upgradenking821
 
Good sql server dba online training
Good sql server dba online trainingGood sql server dba online training
Good sql server dba online trainingsssql
 
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log InsightVMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log InsightVMworld
 
2013 Collaborate - OAUG - Presentation
2013 Collaborate - OAUG - Presentation2013 Collaborate - OAUG - Presentation
2013 Collaborate - OAUG - PresentationBiju Thomas
 
12c In Memory Management - Saurabh Gupta
12c In Memory Management - Saurabh Gupta 12c In Memory Management - Saurabh Gupta
12c In Memory Management - Saurabh Gupta pasalapudi123
 

La actualidad más candente (20)

New Approaches to Faster Oracle Forms System Performance
New Approaches to Faster Oracle Forms System PerformanceNew Approaches to Faster Oracle Forms System Performance
New Approaches to Faster Oracle Forms System Performance
 
Database Performance Tuning Introduction
Database  Performance Tuning IntroductionDatabase  Performance Tuning Introduction
Database Performance Tuning Introduction
 
Sql server 2008r2 dba course
Sql server 2008r2 dba courseSql server 2008r2 dba course
Sql server 2008r2 dba course
 
Analyzing OTM Logs and Troubleshooting
Analyzing OTM Logs and TroubleshootingAnalyzing OTM Logs and Troubleshooting
Analyzing OTM Logs and Troubleshooting
 
DB12c: All You Need to Know About the Resource Manager
DB12c: All You Need to Know About the Resource ManagerDB12c: All You Need to Know About the Resource Manager
DB12c: All You Need to Know About the Resource Manager
 
Oracle applications r12.2, ebr, online patching means lot of work for devel...
Oracle applications r12.2, ebr, online patching   means lot of work for devel...Oracle applications r12.2, ebr, online patching   means lot of work for devel...
Oracle applications r12.2, ebr, online patching means lot of work for devel...
 
How to Turn New Recruits Into Oracle EPM Infrastructure Gurus
How to Turn New Recruits Into Oracle EPM Infrastructure GurusHow to Turn New Recruits Into Oracle EPM Infrastructure Gurus
How to Turn New Recruits Into Oracle EPM Infrastructure Gurus
 
Ten query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowTen query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should know
 
Sql server 2012 dba online training
Sql server 2012 dba online trainingSql server 2012 dba online training
Sql server 2012 dba online training
 
Oracle performance tuning_sfsf
Oracle performance tuning_sfsfOracle performance tuning_sfsf
Oracle performance tuning_sfsf
 
EM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsEM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM Metrics
 
SQL Server Tuning to Improve Database Performance
SQL Server Tuning to Improve Database PerformanceSQL Server Tuning to Improve Database Performance
SQL Server Tuning to Improve Database Performance
 
Windows 7 For Geeks
Windows 7 For GeeksWindows 7 For Geeks
Windows 7 For Geeks
 
Oracle Enterprise Manager 12c - OEM12c Presentation
Oracle Enterprise Manager 12c - OEM12c PresentationOracle Enterprise Manager 12c - OEM12c Presentation
Oracle Enterprise Manager 12c - OEM12c Presentation
 
Getting optimal performance from oracle e-business suite presentation
Getting optimal performance from oracle e-business suite presentationGetting optimal performance from oracle e-business suite presentation
Getting optimal performance from oracle e-business suite presentation
 
Nicholas king oracle epm migration and upgrade
Nicholas king   oracle epm migration and upgradeNicholas king   oracle epm migration and upgrade
Nicholas king oracle epm migration and upgrade
 
Good sql server dba online training
Good sql server dba online trainingGood sql server dba online training
Good sql server dba online training
 
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log InsightVMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
 
2013 Collaborate - OAUG - Presentation
2013 Collaborate - OAUG - Presentation2013 Collaborate - OAUG - Presentation
2013 Collaborate - OAUG - Presentation
 
12c In Memory Management - Saurabh Gupta
12c In Memory Management - Saurabh Gupta 12c In Memory Management - Saurabh Gupta
12c In Memory Management - Saurabh Gupta
 

Similar a Collaborate 2011-tuning-ebusiness-416502

collab2011-tuning-ebusiness-421966.pdf
collab2011-tuning-ebusiness-421966.pdfcollab2011-tuning-ebusiness-421966.pdf
collab2011-tuning-ebusiness-421966.pdfElboulmaniMohamed
 
ebs-performance-tuning-part-1-470542.pdf
ebs-performance-tuning-part-1-470542.pdfebs-performance-tuning-part-1-470542.pdf
ebs-performance-tuning-part-1-470542.pdfElboulmaniMohamed
 
OOW09 Ebs Tuning Final
OOW09 Ebs Tuning FinalOOW09 Ebs Tuning Final
OOW09 Ebs Tuning Finaljucaab
 
Best practices for large oracle apps r12 implementations apps14
Best practices for large oracle apps r12 implementations apps14Best practices for large oracle apps r12 implementations apps14
Best practices for large oracle apps r12 implementations apps14Ajith Narayanan
 
Oracle EBS Production Support - Recommendations
Oracle EBS Production Support - RecommendationsOracle EBS Production Support - Recommendations
Oracle EBS Production Support - RecommendationsVigilant Technologies
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuningOutsourceAX
 
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsOracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsZohar Elkayam
 
SQL 2014 In-Memory OLTP
SQL 2014 In-Memory  OLTPSQL 2014 In-Memory  OLTP
SQL 2014 In-Memory OLTPAmber Keyse
 
Sql server tips from the field
Sql server tips from the fieldSql server tips from the field
Sql server tips from the fieldJoAnna Cheshire
 
(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environmentBIOVIA
 
Surviving the Crisis With the Help of Oracle Database Resource Manager
Surviving the Crisis With the Help of Oracle Database Resource ManagerSurviving the Crisis With the Help of Oracle Database Resource Manager
Surviving the Crisis With the Help of Oracle Database Resource ManagerMaris Elsins
 
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]vasuballa
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersShehap Elnagar
 
DBA 101 : Calling all New Database Administrators (PPT)
DBA 101 : Calling all New Database Administrators (PPT)DBA 101 : Calling all New Database Administrators (PPT)
DBA 101 : Calling all New Database Administrators (PPT)Gustavo Rene Antunez
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersShehap Elnagar
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersShehap Elnagar
 
SQL Explore 2012: P&T Part 1
SQL Explore 2012: P&T Part 1SQL Explore 2012: P&T Part 1
SQL Explore 2012: P&T Part 1sqlserver.co.il
 
cmandtracing-1560359.pdf
cmandtracing-1560359.pdfcmandtracing-1560359.pdf
cmandtracing-1560359.pdfkriole13
 

Similar a Collaborate 2011-tuning-ebusiness-416502 (20)

collab2011-tuning-ebusiness-421966.pdf
collab2011-tuning-ebusiness-421966.pdfcollab2011-tuning-ebusiness-421966.pdf
collab2011-tuning-ebusiness-421966.pdf
 
ebs-performance-tuning-part-1-470542.pdf
ebs-performance-tuning-part-1-470542.pdfebs-performance-tuning-part-1-470542.pdf
ebs-performance-tuning-part-1-470542.pdf
 
ADF Performance Monitor
ADF Performance MonitorADF Performance Monitor
ADF Performance Monitor
 
OOW09 Ebs Tuning Final
OOW09 Ebs Tuning FinalOOW09 Ebs Tuning Final
OOW09 Ebs Tuning Final
 
Best practices for large oracle apps r12 implementations apps14
Best practices for large oracle apps r12 implementations apps14Best practices for large oracle apps r12 implementations apps14
Best practices for large oracle apps r12 implementations apps14
 
Oracle EBS Production Support - Recommendations
Oracle EBS Production Support - RecommendationsOracle EBS Production Support - Recommendations
Oracle EBS Production Support - Recommendations
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuning
 
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsOracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
 
Breaking data
Breaking dataBreaking data
Breaking data
 
SQL 2014 In-Memory OLTP
SQL 2014 In-Memory  OLTPSQL 2014 In-Memory  OLTP
SQL 2014 In-Memory OLTP
 
Sql server tips from the field
Sql server tips from the fieldSql server tips from the field
Sql server tips from the field
 
(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment
 
Surviving the Crisis With the Help of Oracle Database Resource Manager
Surviving the Crisis With the Help of Oracle Database Resource ManagerSurviving the Crisis With the Help of Oracle Database Resource Manager
Surviving the Crisis With the Help of Oracle Database Resource Manager
 
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]
OOW16 - Getting Optimal Performance from Oracle E-Business Suite [CON6711]
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powers
 
DBA 101 : Calling all New Database Administrators (PPT)
DBA 101 : Calling all New Database Administrators (PPT)DBA 101 : Calling all New Database Administrators (PPT)
DBA 101 : Calling all New Database Administrators (PPT)
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powers
 
T sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powersT sql performance guidelines for better db stress powers
T sql performance guidelines for better db stress powers
 
SQL Explore 2012: P&T Part 1
SQL Explore 2012: P&T Part 1SQL Explore 2012: P&T Part 1
SQL Explore 2012: P&T Part 1
 
cmandtracing-1560359.pdf
cmandtracing-1560359.pdfcmandtracing-1560359.pdf
cmandtracing-1560359.pdf
 

Último

UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdfUGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdfNirmal Dwivedi
 
Single or Multiple melodic lines structure
Single or Multiple melodic lines structureSingle or Multiple melodic lines structure
Single or Multiple melodic lines structuredhanjurrannsibayan2
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.pptRamjanShidvankar
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - Englishneillewis46
 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17Celine George
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...pradhanghanshyam7136
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptxMaritesTamaniVerdade
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024Elizabeth Walsh
 
Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsKarakKing
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentationcamerronhm
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxAreebaZafar22
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.MaryamAhmad92
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.christianmathematics
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxVishalSingh1417
 
How to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSHow to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSCeline George
 
How to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptxHow to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptxCeline George
 
Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseAnaAcapella
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxEsquimalt MFRC
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsTechSoup
 
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfUnit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfDr Vijay Vishwakarma
 

Último (20)

UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdfUGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
 
Single or Multiple melodic lines structure
Single or Multiple melodic lines structureSingle or Multiple melodic lines structure
Single or Multiple melodic lines structure
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - English
 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
 
Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptx
 
How to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSHow to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POS
 
How to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptxHow to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptx
 
Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please Practise
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfUnit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
 

Collaborate 2011-tuning-ebusiness-416502

  • 1. 1
  • 2. <Insert Picture Here> Tuning All Layers Of E-Business Suite – Performance Topics Isam Alyousfi, Senior Director Lester Gutierrez, Senior Architect, EBS Applications Performance
  • 3. 3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  • 4. 4
  • 5. 5 Objectives • Overview of Performance Methodology for Optimizing E- business suite • Outline best practices for Tuning E-Business layers – Tuning the Applications Tier – Tuning the Concurrent Manager – Tuning the Database Tier – Tuning the Applications
  • 6. 6 R12 Apps Basic Layers / Products B r o w s er Lis ten er mod_oc4j mod_proxy mod_plsql mod_osso Apache JVM1 App1 App2 OC4 J oc4j_security LDAP Oracle Forms Repository OID RAC Database + db Listener IdM JVM2 App3 App2 Concurrent Processor, Manager Ext. procs OAM
  • 7. 7 R12 Apps Basic Layers / Products
  • 8. 8 Application & SQL Management System Resource Management Space Management Backup & Recovery Management Storage Management Database Management Procedural stmts. Procedural Statement ExecutorSQL stmts. SQL Statement Executor PL/SQL Block PL/SQL engine SQL engine CPU SGA Memory SQLNET JDK OC4J JAVA Forms JDBC OAFramework XML BI Publisher Buffer Cache WF Concurrency Discoverer Optimizer Access Path Indexes 600,000+ SQL statements 200 + Products FND, GL, AP, AR, OM, SC, … Concurrent manager MLS 100,000 + PLSQL packages 40,000 + Tables OJSP OCI SOA BPEL Out of Memory
  • 9. 99 Performance Methodology for Optimizing of E-business suite • How to Approach a Performance Issue • Where is the time going ? Get the right diagnostics – Where? – What? – Why?
  • 10. 1010 How to Approach a Performance Issue 1. DEFINE the problem clearly 1. GATHER the right data to analyze the issue 1. Identify the ROOT CAUSE of the problem, possibly gather additional data 1. Search for a KNOWN SOLUTION or workaround that addresses the root cause of the problem 1. If it is a product issue, PASS ON the right information to support/dev through the regular channels 1. Try to identify a TEMPORARY WORKAROUND to alleviate the issue while you get a product fix
  • 11. 1111 Define the problem • Get a clear understanding of the performance issue What Where When Why How
  • 13. 1313 Gather data • Start with Best Practices : (note: 1121043.1) • SQL Tuning – Trace files – SQLT output (note: 215187.1) – Trace Analyzer (note: 224270.1) – AWR Report (note: 748642.1) – AWR SQL Report (awrsqrpt.sql) – 11g SQL Monitoring – SQL Tuning Advisor • PL/SQL Tuning – Product logs – PL/SQL Profiler (note: 808005.1) • Reports Tracing • note: 111311.1 •D atabase Tuning • AWR Report (note: 748642.1) • ADDM report (note: 250655.1) • Automated Session History (ASH) Report • LTOM output (note: 352363.1) •F orms Tuning • Forms Tracing (note: 373548.1) • FRD Log (note: 445166.1) • Generic note: 438652.1 •M iddletier Tuning • JVM Logs • JVM Sizing (note: 362851.1) • JDBC Tuning (note: 278868.1) •O
  • 14. 14 Root Cause Analysis • Where is the time going? • Who is consuming time? • Why?
  • 15. 15 Where is the time going ? Get the right diagnostics • Identify the components that do the processing – Refer to the architecture • which tier(s) play a part ? • which components within a tier come into play ? • 80% of issues are DB related
  • 16. 16
  • 17. 17 Root Cause Analysis: Check the basics • Version levels • Required Configuration – Init.ora : Notes 396009.1 (R12) & 216205.1 (11i) • Latest patch levels • Recommended Patch lists – Note “Recommended Performance Patches for the Oracle E-Business Suite”, 244040.1 – Also check for product specific notes • Available Fixes • Best Practices in Performance for EBS – http://blogs.oracle.com/stevenChan/
  • 18. 18 Where is the time going ? Get the right diagnostics • If non-database (client side) processing occurs within the flow that was traced, you may see wait events like – SQL*Net Message% – TCP Socket% – PLSQL Lock Timer • Other useful data in identifying where time is spent – Raw sql trace file – Product logs – App server logs such as http logs – Client process cpu consumption from OS (for example ps)
  • 20. 20 Tuning the Applications Tier •Forms –To reduce load on the database server, ensure users are optimally utilizing the professional Forms interfaces. –Avoid Blind queries –Provide selective criteria in Find windows and LOVs –Minimize network traffic and form open times –Avoid opening and closing forms across transactions –Combine forms from multiple-products onto a single menu –Generate complete SQL trace and Forms Runtime Diagnostics (FRD) to debug performance issues
  • 21. 2121 Where is the time going ? Get the right diagnostics • Client Process CPU Consumption – Help -> Oracle Applications – Forms and Database process id & CPU Time
  • 22. 22 Tuning the Applications Tier OC4J/JVM •Use one JVM per 2 CPUs •~100 concurrent users per JVM
  • 23. 23 Tuning the Applications Tier OC4J •Response Time/CPU Usage •OutOfMemoryErrors
  • 24. 24 Tuning the Applications Tier •First, rule out any SQL issues. –Note 357597.1 on how to enable SQL trace for OA Framework applications •If there are no database-related issues, then –you need to analyze the JVM •Techniques you can use: –Thread dumps – Send to Oracle Support for analysis –GC logs: Configure JVM sizing –Profilers - Send to Oracle Support for analysis Response Time/CPU Issues
  • 25. 25 Tuning the Applications Tier •GC tuning –Total available memory is the most important factor affecting GC performance. –Review the frequency of collections, especially major collections (i.e. Full GC) –Enable verbose GC to tune heap sizes based on the GC traffic –Start with: -Xms1024M and –Xmx1024M –Setting -Xms and -Xmx to the same value avoid the need to memory allocation during runtime –If full GCs are too frequent, consider increasing Xms and Xmx
  • 26. 26 Tuning the Applications Tier •Common causes of OutOfMemoryError –Memory Leak – Some objects not cleaned up •Strong reached references, connection objects, etc. –Memory Hemorrhage – •Some kind of infinite loop •Reading large data sets consuming vast amounts of memory in a short period of time •Connection leaks leading to vast object allocations •SQL Statements/caching may lead to excess memory used in JDBC libraries –Sizing Issues – Heap is undersized for expected load
  • 27. 27 Tuning the Applications Tier •Symptoms of memory leak –Heap usage increases steadily, over a relative longer period of time (e.g. serveral hours or days) –When the amount of memory leak is large enough, you will start seeing continuous Full GC in the GC log –Different users may encounter OOM on totally unrelated flows –Generate Heap Dump files when OutOfMemoryError occurs •Help Oracle Support identify the root cause of the leaks
  • 28. 28 OA Fwk : JVM Process ID Process ID of the JVM handling your session • Client Process CPU Consumption • OA FWK – JVM Process ID • The “About” link is at the bottom of each page • Requires FND_DIAGNOSTICS profile set to
  • 29. 29 <Insert Picture Here> Tuning the Concurrent Manager
  • 30. 30 Tuning the Concurrent Manager –To maximize throughput for jobs which spawn parallel workers (i.e. Auto Invoice, Payroll), consider reducing the sleep time of the Conflict Resolution Manager (CRM) • Default is 60s, consider 5 or 10 seconds –Increase the cache size (number of requests cached) to at least twice the number of target processes –If the CM and Database are co-located, use IPC or bequeath connection –Consider dedicating certain concurrent managers to process either short or long running programs to avoid queue backup –Purge the FND tables on a regular basis using the “Purge Concurrent Request and Manager Data” program
  • 31. 31 Tuning the Concurrent Manager •Workload Management –Trim the fat. Review and eliminate concurrent jobs that are not required and/or are not being used by your users. –Avoid enabling an excessive number of standard or specialized managers –A common guideline is between 1-2 target process per CPU, but this needs to be balanced with resources required by online activity
  • 32. 32 Tuning the Concurrent Manager •Workload Management –Use specialization rules and work shifts to bind specific jobs to specific time windows •Helps avoid scheduling resource intensive batch requests during peak activity –Reschedule some programs to run when the concurrent managers have excess capacity and –Add more queues if all queues are running at maximum capacity only if resources permit.
  • 33. 33 Tuning the Concurrent Manager •Transaction Managers - TMs –Used for synchronous online processing(ex:Inventory Transactions) –Ensure enough TMs exist to service the request load •Set the profile “Concurrent:Wait for Available TM” to 1 (second). –Set the sleep time on the TMs to a high number (e.g. 10 minutes) •Avoids constant polls to check for shutdown requests
  • 34. 34 <Insert Picture Here> Tuning the Client & Network
  • 35. 35 Tuning the Client –Minimize the browser memory footprint •Minimize toolbars, browser themes, extensions, plug-ins, helper applications, and phishing controls •Do not minimize E-Business Suite components (but minimize other apps) •If there is a choice, use OAF/HTML (smaller memory profile) –Upgrading RAM is generally more important than CPU –Always keep menus small (customize if needed) •Menus are both memory and network intensive –Oracle E-Business Suite 11i and 12 PC Client Performance •http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html
  • 36. 36 Tuning the Network –Routing Issues –Lost Packets due to device saturation –DB and MT must be co-located on a local private network –Traffic from MT to Client: •Consider using Local proxy in local offices (global single instance), (note 153287.1)
  • 38. 38 Tuning the Database Tier – Configuration • Ensure mandatory init.ora parameters are set correctly – MOS Docs 396009.1 & 216205.1 (R12 & 11i respectively) – Use AWR advisories to optimize SGA/PGA sizing • Required Patches & Fixes/Workarounds for Known Issues – Recommended Performance Patches -MOS Doc 244040.1 – Required Patching – Interoperability Notes •11gR1 Certified with 11i and R12 – MOS Docs 452783.1 & 735276.1 •11gR2 Certified with 11i – MOS Doc 881505.1 •10gR2 Uptake patch bundle 8362683, MOS Doc 791049.1
  • 39. 39 Tuning the Database Tier – Configuration • Use key Database features – Auto Memory Management • Exception – Linux HugePages – MOS Doc 744769.1 – System Managed Undo – Auto Segment Space Management – PL/SQL Native Compilation • Convert to the OATM Tablespace Model – Consolidates # of Apps tablespaces from 400 to ~12 • MOS Doc 248857.1 – Best practices: LMT, ASSM, groups objects by size/type/access pattern – Migration utility can be used to migrate a schema at a time – Use AUTOALLOCATE extent sizes
  • 40. 40 Tuning the Database Tier – Disk IO • I/O Optimization – Configure & verify direct IO is used if using file system datafiles • Improves performance/scalability by simulating raw devices, avoiding file system cache traffic – Ensure your IO subsystem can handle your peak IO load & IOPS • RAID, S.A.M.E. configs with sufficient spindles • Measure with ORION tool or in 11g DBMS_RESOURCE_MANAGER.CALIBRATE_IO – MOS Doc 727062.1 – Reduce the IO workload • Tune expensive SQL • Trim the workload of unnecessary tasks • Do 3X less IO on average with 11g Advanced Compression Option
  • 41. 41 Tuning the Database Tier – Statistics Gathering • Use only FND_STATS or the Gather Schema / Table Statistics Concurrent Programs – Do NOT USE the analyze or dbms_stats command directly. It is not supported, and can result in sub-optimal plans • Use the GATHER_AUTO option to gather incrementally • Analyze all schemas at 10%, then specific objects at x% – Due to data skew, some tables benefit from higher sampling % – For 11g, can use value of zero (AUTO_SAMPLE_SIZE) • Disable the 10g/11g automatic job to gather stats
  • 42. 42 Tuning the Database Tier – Statistics Gathering • Consider locking stats for very volatile tables once a representative set is gathered – Some Interface tables – Other examples: ap_selected_invoices, wsh_pr_workers – Use FND_STATS.LOAD_XCLUD_TAB to skip the table • Do not gather statistics excessively on entire schemas or the entire database such as nightly or weekly • FND_STATS now supports Database 11g Extended Stats. Patch 9470196. Also, in patch set 9962815
  • 43. 43 Tuning the Database Tier – Performance Management • Establish baselines for different workloads –Use AWR baselines & diff reports •DBMS_WORKLOAD_REPOSITORY or Enterprise Manager •OS metrics with tools such as OS Watcher – MOS Doc 301137.1 –Timings/Traces for critical flows & concurrent programs • Use 11g’s SQL Performance Analyzer (SPA) & SQL Plan Management features (SPM) –Change management performance impact –Stabilize execution plans as needed & quickly restore “good” plans while analysis and permanent solutions are found
  • 44. 44 Tuning the Database Tier - AWR • Regularly Review Automatic Workload Repository data • Review Top 5 Timed Events section to determine utilization & bottlenecks and review relevant detailed sections for these
  • 45. 45 Tuning the Database Tier - AWR • For latch related waits – Often due to non-sharable SQL, sub-optimal SQL which performs full table or full index scans, dynamic object creation/removal, etc. – Review the latch Statistics section to determine the hot latches – Trace some waiter and holder sessions to determine actual cause • For I/O related waits – Review SQL sections by Logical/Physical reads and the Segment Statistics sections by I/O • For enqueue related or buffer busy waits – Review the following sections: enqueue, segment statistics by buffer busy waits, row lock & ITL waits
  • 46. 46 Tuning the Database Tier - AWR • Link AWR with ASH data – Map sessions & SQL to main bottlenecks – Query Active Session History (ASH) data – Enable tracing for a few of the sessions with heavy waits determine the actual cause and SQL statements. – Review the Advisory sections in AWR to fine tune SGA & PGA – Correlate top SQL in AWR to business flows – Produce and review SQL AWR reports for top SQLs (awrsqrpt.sql) – Monitor OS Stats (included in 11g AWR) • CPU and Memory Utilization & Paging, I/O statistics & response times
  • 47. 47 Tuning the Database Tier – DB Console
  • 48. 48 Tuning the Database Tier – DB Console
  • 49. 49 • If bottleneck is IO related... (db file sequential read, db file scattered read, log file sync, free buffer waits, etc...) – Tune Top SQL is always the first step – Maximize Memory availability, allocate generously to buffer cache, review AWR advisories & monitor swapping & paging – Use ASM or alternatively use the SAME methodology for db files – IO Sub-system: • RAID 10 still preferred most often for high end requirements Ideal avg. response times of < 10ms. – Check for excessive/redundant indexing – Configure async IO, use quick IO technologies Tuning the Database Tier – Common corrective actions to maximize scalability
  • 50. 50 • If the bottleneck is concurrency related (enq %, latch%, buffer busy%, etc ) – Use global hash-partitioned indexes for hot leaf blocks - Identify via AWR "Top logical IOs by Segment" – Some of these waits are commonly caused by bad SQL execution plans – i.e. :latch: cache buffer chains, %buffer busy waits, read by other session – Make sure to use ASSM and OATM tablespace model. – Increase INITRANS to alleviate ITL contention – Increase sequence caches – Work with support/consulting to evaluate if table/index partitioning will help – If the program allows, more frequent commits (batch size) can help by reducing the CR (consistent read) work required Tuning the Database Tier – Common corrective actions to maximize scalability
  • 51. 51 • If the bottleneck is RAC related ... (gc %, global%, etc...) – First do all the steps mentioned previously... – Check interconnect configuration -- Use Jumbo frames. – Do not use parallel query/dml across nodes • Use instance_groups & parallel_instance_groups settings – Review Parallel Concurrent Processing & Application Affinity config • 12.1.3 introduced option to define node affinity at the program level – Concurrent -> Program -> Define – Perform RAC focused AWR analysis … Tuning the Database Tier – Common corrective actions to maximize scalability
  • 52. 52 Tuning the Database Tier – Common corrective actions to maximize scalability • RAC AWR Analysis 1.Verify SQL execution is optimized – 10046 trace/SQL Monitoring 2.Collect AWR data from all instances - awrgrpt.sql and awrgdrpt.sql (cluster-wide base and diff reports) 1.Verify IO, CPU utilization and Interconnect messaging performance are within acceptable limits 2.Drill down on cluster wait events + “SQL order by Cluster Waits” 3.Correlate SQL executions on all instances accessing/modifying top segments in Segment Statistics by • Global Cache Buffer Busy & Buffer Busy Waits • CR/Current Blocks Received • Logical Reads
  • 53. 53 • To get full value from tracing… – Raw trace must be • A complete, non-truncated trace file *** DUMP FILE SIZE IS LIMITED TO 12345 BYTES*** • From before the user action began To the point the DB session ends** – Tkprof output MUST have • Runtime Execution Plans and Row Counts & Row Source Stats • Run Time Execution Plan = “Row Source Operation” != “Explain Plan” – Best practices, detailed how-to steps & recommended tracing patches documented • MOS Doc 1121043.1 -Also review Appendix B – Getting In-Memory partial runtime stats Tuning the Database Tier – TKPROF: Good, Bad & Ugly
  • 54. 54 Tuning the Database Tier – Good TKPROF call count cpu elapsed disk query current rows ------- ------ -------- ---------- ---------- ---------- ---------- ---------- Parse 1 0.00 0.00 0 0 0 0 Execute 7674758 780.79 728.70 0 1 0 0 Fetch 7674758 6328.78 6757.80 225987 153846532 0 7674758 ------- -------- -------- ---------- ---------- ---------- ---------- ---------- total 15349517 7009.58 7486.52 225987 153846533 0 7674758 Misses in library cache during parse: 1 Misses in library cache during execute: 1 Optimizer mode: ALL_ROWS Parsing user id: 173 (recursive depth: 2) Rows Row Source Operation ------- --------------------------------------------------- 7674758 SORT AGGREGATE (cr=153846532 pr=225987 pw=0 time=7174772779 us) 7675089 NESTED LOOPS (cr=153846532 pr=225987 pw=0 time=6919378560 us) 7675089 SORT UNIQUE (cr=123146176 pr=162809 pw=0 time=6083080435 us) 55878643 TABLE ACCESS BY INDEX ROWID AR_TRX_COGS_GT (cr=123146176 pr=162809 pw=0 time=5577872837 us) 136390387 INDEX RANGE SCAN AR_TRX_COGS_N1 (cr=23439707 pr=37706 pw=0 time=2136416980 us)(object id.. 7675089 TABLE ACCESS BY INDEX ROWID RA_CUSTOMER_TRX_LINES_ALL (cr=30700356 pr=63178 pw=0 time=539908369 us) 7675089 INDEX UNIQUE SCAN RA_CUSTOMER_TRX_LINES_U1 (cr=23025267 pr=2820 pw=0 time=196155435 us)(object ..
  • 55. 55 Tuning the Database Tier • Why upgrade EBS to – Internal tests show 10-20% improvement in EBS batch & reporting workloads – Significant RAC Performance improvements – Oracle internal’s production EBS system on 11gR1; •11gR2 scheduled for the summer – Lower cost with Advanced compression, better/automated diagnostics, lower-risk upgrade with SPM/RAT/SPA … – Best Practices/Tips white papers •twp_upgrading_10g_to_11g_what_to_expect_from_optimizer.pdf •http://www.oracle.com/apps_benchmark
  • 56. 56 11g SQL Tracing Enhancements – New tracing parameter PLAN_STAT • first_execution | all_executions | adaptive* – Connection pool problem to get row source stats is solved – Key tracing patches* listed on MOS Doc 1121043.1 – Turn trace on for just 1 or a list of sql_id’s – With real-time row source stats monitoring from v$sql_plan_statistics – no more waiting hours for a complete trace to get the right trace data – Expanded event syntax : - alter session set events ’sql_trace [sql:a5ks9fhw2v9s1|56bs32ukywdsq] … - sql_trace wait=true | false, bind=true | false,planstat=never | first_execution | all_executionss|adaptive level = 1| 4| 8| 12 | 16
  • 57. 57 11g Automatic SQL Monitoring & Tuning
  • 58. 58 11g Performance Enhancements • Optimizer improvements – SPM (SQL Plan Management)– quick relief while permanent solution arrives – Better density estimates for skewed data – Faster, more accurate statistics with AUTO_SAMPLE_SIZE – Extended statistics - address predicate correlation – Adaptive cursor sharing – to address bind peeking issues • Some others… – Interval partitioning, Additional partitioning schemes – Text indexing performance enhancements – RAC scales better & AWR is more RAC aware – Many others..
  • 59. 59 11g Advanced Compression • Data compression for all data types • Structured, Unstructured, Backup, Network • Reduces storage consumption by 2 to 4 times • Improves read performance • Enhances memory, buffer cache utilization • Complete application transparency • Benefits diverse application workloads Initially Uncompressed Block Compressed Block Partially Compressed Block Compressed Block Empty Block
  • 60. 60 • Oracle Database 11g extends table compression for OLTP data – Support for conventional DML Operations (INSERT, UPDATE) – New algorithm significantly reduces write overhead • Batched compression minimizes impact on OLTP transactions – No impact on reads • Reads data directly in its compressed format • Many reads actually see improved performance due to fewer I/Os and enhanced memory efficiency 11g Advanced Compression
  • 61. 61 Leaf Blocks Nancy,ROWID Nancy,ROWID Nancy,ROWID Nicole,ROWID Norm,ROWID Pablo,ROWID Paula,ROWID Paula,ROWID Peter,ROWID Luis,ROWID Mark,ROWID Mary,ROWID Mike,ROWID Mike,ROWID Karl,ROWID Kathy,ROWID Kim,ROWID Lance,ROWID Pierre,ROWID Rachel,ROWID Rajiv,ROWID Raoul,ROWID Phil,ROWID F H Kar N P Ph B C Cr Sam St Su Di Lu Rh 11g Advanced Compression – Index Compression • Compress multi-column indexes to reduce the storage overhead of repeated values • Stores more keys per leaf block • Less I/O,better performance of range scans
  • 62. 62 11g Advanced Compression - Compression Advisor • Estimate Storage Savings for your case – Available in 11g Release 2 or … – Available on OTN * for previous releases – Shows projected compression ratio for uncompressed tables – Reports actual compression ratio for compressed tables (11g Only) * http://www.oracle.com/technology/products/database/compression/compression-advisor.html
  • 63. 63 •Oracle’s Internal E-Business Suite Production system deployed ACO in ‘09 –Average overall storage savings - 3x •Table compression 4x •Index compression 2x •LOB compression 2.3x –231TB of realized storage savings primary, standby & test systems •Additional benefits also accrued in Dev clones and backup •Payroll, Order-2-Cash, AP/AR batch flows, Self-Service flows run without regression, Queries involving full table scans show speedup 11g Advanced Compression with E-Business Suite
  • 64. 64 • White paper: Upgrade to Oracle 11g - Performance Best Practices http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html • Special attention to – Testing of 11g or 10g workloads with Real Application Testing – Managing SQL Execution Plans and minimizing performance degradation with11g SQL Plan Management 11g Upgrade Best Practices for EBS
  • 66. 66 Tuning the Applications • MyOracleSuport note “Recommended Performance Patches for the Oracle E-Business Suite”, 244040.1 – Recommended performance patches for all the modules and tech. stack components are consolidated in this note • Apply latest Release Update Packs • Functionality, fixes, and updates for all Release 12 functional applications and the technology stack • Several product family RUPs included are also available individually
  • 67. 67 Tuning the Applications • Apply latest Release Update Packs – Release 12.1.3: Patch 9239090 • MOS Doc: 1080973.1 – Release 12.1 : Patch 7303030 • MOS Doc: 752619.1 – Release 12.0 - RUP6: Patch 6728000 • MOS Doc: 743368.1 • Review EBS Benchmark Publications • http://www.oracle.com/apps_benchmark/html/white-papers-e-business.html
  • 68. 68 Tuning the Applications • Performance Best Practices – Workflow • For background engines via Concurrent Manager, set the ‘Process Stuck’ parameter to ‘No’ – Start a separate background engine to handle stuck processes with a low frequency –i.e. once or twice a day • Use deferred activities to improve online response times for flows such as Scheduling, PO Document Approval, etc. • Frequently purge runtime data (Conc Program: FNDWFPR) • Run $FND_TOP/sql/wffngen.sql to translate the activity function calls into static calls • Disable retention on Workflow Queues – DBMS_AQADM.ALTER_QUEUE(queue_name=>:b1,retention_time=>0); • For high volume batch processing in RAC use ITEM_TYPE to Node affinity
  • 69. 69 Tuning the Applications • Performance Best Practices – Purge & Archive • Implement a regular purge process including transient data as well as obsolete transactional data • Purge old data before performing upgrades • Many documented standard programs available – ~260 purge programs in R12 • Purge Portal – Purge programs can be configured, initiated and monitored • Set the execution frequency as well view history of purge programs – R12: programs tagged with the “Purge” program type
  • 70. 70 • Runtime Performance Testing Tips – Use Automated, scripted tools • EBS Test Started Kits (Winrunner/QTP) – Bundled QA based automated scripts for EBS testing - Patch 8408886 • Oracle Applications Testing Suite (Accelerators for EBS) – Web and Forms based flows – Complement with user participation tests and batch load tests with frequent and critical jobs – References http://blogs.oracle.com/stevenChan/2009/10/oats_ebs_certified.html http://blogs.oracle.com/stevenChan/2009/08/evolutionary_steps_automated_testing_ebs.html Tuning the Applications
  • 72. 72 References • R12.1 documentation roadmap (790942.1) • “Oracle E-Business Suite Release 12.1 Info center” (806593.1) • Database preparation guidelines for R12.1 upgrade (761570.1) • Patching FAQs (459156.1, 225165.1) • Using staged or shared APPL_TOP and distributed AD (734025.1, 384248.1, 236469.1) • OAM “Patch Wizard” overview and FAQ (976188.1, 976688.1) • AD Command Line Options for Release R12 (1078973.1) • EBS 12.1.3 Data Model Comparison Report (1290886.1) • Recommended Performance Fixes (244040.1) • R12 Upgrade Sizing & Best Practices (399362.1)
  • 73. 73 SUMMARY DO’s.. •Read and follow official documentation and have a project plan •Engage Support early on to validate your upgrade plan •Identify and execute tasks you can do today •Test ! Test ! Test ! •Keep track of patches applied in test upgrades •Be smart about using the right tools and explore published downtime reduction techniques •Optimize patch runs to suite your H/W DON’Ts.. •Ignore including relevant functional SMEs in the planning process from the beginning •Skip failed or long running jobs as a regular practice •Drop indexes •Run adpatch in serial mode •Perform workarounds without approval from support/DEV •Insufficient time and/or # of rounds of upgrade and critical functional flow testing
  • 74. 74 Oracle Products Available Online Oracle Store Buy Oracle license and support online today at oracle.com/store
  • 75. 75
  • 76. 76
  • 78. 78

Notas del editor

  1. What is slow ? Single process or screen or user(s) Everything in a module Entire instance Where is it slow ? Test / Dev / Production / All When is it slow ? Is it slow at particular times of the day ? Is it slow when there are only a few users on the system or only why the system is loaded ? Since when is this slow ? What could have changed since the last time it ran ok ? Patches,Gather stats,,Change in Data Volumes, etc Why is it slow ? Why do you think it is slow? May be customer will tell they have applied patch, or recent load etc How ? How to reproduce the issue?
  2. What is slow ? Single process or screen or user(s) Everything in a module Entire instance Where is it slow ? Test / Dev / Production / All When is it slow ? Is it slow at particular times of the day ? Is it slow when there are only a few users on the system or only why the system is loaded ? Since when is this slow ? What could have changed since the last time it ran ok ? Patches,Gather stats,,Change in Data Volumes, etc Why is it slow ? Why do you think it is slow? May be customer will tell they have applied patch, or recent load etc How ? How to reproduce the issue?
  3. Trace File set max_dump_file_size = unlimited AND statistics_level = all Verify row source Verify total time Compare time reported in trace vs customer reported time For parallel queries, pick files from udump AND from *bdump* directory Interpreting Raw SQL_TRACE: Note 39817.1 SQLT (note 215187.1) set statistics_level = all methods with order of preference: ..sqltxecute.sql ..sqltxtract.sql - use sql_id verify row counts if run while query is running SQL Tuning Advisor is invoked. In 11g, SQL Monitoring report will be produced. ..sqltxplain.sql AWR Report (748642.1) create snapshots...dbms_workload_repository.create_snapshot() generate report: $ORACLE_HOME/rdbms/admin/awrrpt.sql AWR sql report: awrsqrpt.sql ASH Report get using $OH/rdbms/admin/ashrpt.sql or ashrpti.sql ADDM Report (250655.1) create awr snapshots $ORACLE_HOME/rdbms/admin/addmrpt.sql Trace Analyzer if trace file is not too big PL/SQL Profiler https://webiv.oraclecorp.com/cgi-bin/webiv/do.pl/Get?WwwID=note:808005.1 Events alter system set events &amp;apos;4030 trace name errorstack level 3&amp;apos;; Dumps &amp;quot;System state: alter session set events &amp;apos;immediate trace name SYSTEMSTATE level 10&amp;apos;;  -- for hang issue ST support asked us to produce with level 266&amp;quot; Process state: alter session set events &amp;apos;immediate trace name PROCESSSTATE level 10&amp;apos;; Solaris...truss -a -e -f -rall -wall -o /tmp/truss.out -p &amp;lt;UNIX pid&amp;gt; Linux...strace -p &amp;lt;UNIX pid&amp;gt; 11g Real Time SQL Monitoring http://structureddata.org/2008/01/06/oracle-11g-real-time-sql-monitoring-using-dbms_sqltune.report_sql_monitor/ OSWatcher: 301137.1 Forms FRD log and Forms Trace location: $FORMS_TRACE_DIR Frd log file: collect_&amp;lt;pid&amp;gt; Forms trace file: forms_&amp;lt;pid&amp;gt;.trc Performance Tuning Assistant: http://unique.us.oracle.com:7777/PTAv3/
  4. Where…in which component Who…SQL, PL/SQL, waits or unaccounted waits…further drilldown to objects Why…5 whys concept
  5. -train forms users selective searches , no blind queries avoid opening/closing forms across trxs
  6. -- if you disable hotspot/jit ..we’ll be slower due to always be running in interpreted mode Print GC Details = infor on individual heap area sizing/gc activity Minor &amp; Major Collections Elapsed Time of GC Can use the GC Timestamp and GC Details to print additional information (JDK 1.4 or higher): -XX:+PrintGCTimeStamps -XX:+PrintGCDetails 0.000: [GC [PSYoungGen: 43008K-&amp;gt;3698K(50176K)] 43008K-&amp;gt;3698K(160768K), 0.0770625 secs] 1.483: [GC [PSYoungGen: 46706K-&amp;gt;3786K(50176K)] 46706K-&amp;gt;3786K(160768K), 0.0748696 secs] 2.903: [GC [PSYoungGen: 46794K-&amp;gt;3784K(50176K)] 46794K-&amp;gt;3784K(160768K), 0.0115077 secs] Verbose GC Output Location $LOG_HOME/ora/10.1.3/opmn/OC4J~&amp;lt;coregroupname&amp;gt; ~default_group~&amp;lt;core#&amp;gt; Example: OC4J~oacore~default_group~1
  7. if GC is happening too frequently – you can increase heapsize
  8. -Sun JDK options listed enable auto heap dmp generation set file location/name -Other JDKs in “long version” OOW DVD content Add –XX:+HeapDumpOnOutOfMemoryError –XX:HeapDumpPath=/tmp to the JVM startup options to generate a heap dump on OOM in the specified path
  9. -Workload management # of queues, target processes, work shifts, trim fat -CRM sleep time set to null or 10
  10. TM’s -sync, online processing -configure to reduce waits “Concurrent:Wait for Available TM “ = # of seconds the client will wait for a TM to be available before moving on to try another -use queues in RAC – allows failover setup ; single instance can still use pipes and some custs have reported better perf. -----------------------------------------------
  11. Some tip highlights reduce browser memory footprint get rid of toolbars,themes, plug-ins,phising controls that are not REQUIRED if there is a choice, use OAF/HTML screens more often – or use higher-spec PCs for forms Ram upgrade will give you more benefit than cpu boosts generally customize menus if needed to keep them small to save on memory and nw load ----------------------------------------------------------
  12. routing issues losing ACKS and therefore there could be a request to retransmit the entire TCP window again quality of the line (excessive capacitance/noise )
  13. -Comply with parameter setting requirements -buffer cache and shared pool sizing specially critical - a reference sizing table is included in the above notes - can also use advisory features to adjust
  14. We recommend you convert to OATM - optional in 11i and R12, but R12 new objects get created in OATM tablesspaces 400 tbspaces to ~ 12 better manageability, performance, reclaim space, storage best practices pre-configured
  15. go for async IO Go for Quick IO for file system dbf’s Temp tablespace = locally managed, uniform size, type=temp Review AWR reports
  16. don’t over-gather ..do mostly deltas (GATHER AUTO) Avoid gathering during peak activity
  17. Here we see an example of a system with a “little” contention !
  18. SQL Identification New SQL_ID is used in place of SQL hash value to uniquely identify SQL statements Preserved across releases Allows SQL statement statistics to be easily compared across upgrades.
  19. DB Console A component of 10g Grid Control Complete Management Portal Alerts Maintenance Performance Administration Availability Space Management SQL Tuning Advisor
  20. Keep up-to-date with the recommended performance patches -the note listed includes techstack and product specific patches Apply the latest Release Update Packs for R12 it is RUP2, released last month (techstack only version also available)
  21. Keep up-to-date with the recommended performance patches -the note listed includes techstack and product specific patches Apply the latest Release Update Packs for R12 it is RUP2, released last month (techstack only version also available)
  22. Handling stuck processes WF backgroung engines – set ‘Process Stuck’ param to NO Use a separate bckground engine for stuck processes – with low frequency Use deferred activities where possible to speed up online response times Scheduling for example Purge regularly WF runtime data – it grows fast, Some customers purge as much as twice a day
  23. Purge &amp; Archive Oracle is committed to providing Purge/Archive solutions for the key transactional entities The native E-Business Suite purge and archive programs are synchronized with the data model implement regular, on-going process &amp; policies Several product specific Purge FAQ’s/Instrunctions metalink notes exist – review them Purge before upgrade Use the Purge Portal in OAM to manage your Purge work – configure, launch, monitor, verify With new Purge program type, you can easily set up a specific CM queue just for purging &amp; configure shifts, etc. -------------------------------------------------