These are the warmup slides for the Wildcard 13 conference (Riga, Latvia, September 13th.2013
Join the discussion with oracle professionals, get your problems solved and help others! Bring your questions and problems with you to discuss them in a larger group of oracle professionals. We'll discuss anything you have related to Oracle Databases - performance tuning, coding standards and instrumentation, configuration issues, database design, migration strategies, system architectures, upgrade issues, etc.
The chances are:
The question will be answered or the problem will be solved.
You'll have more ideas to explore and try to address the issue.
You'll spend fun time helping others by sharing your experience.
You'll get a free beer for your courage to join the discussion.
29. I/O Bottleneck Problem
Response Time Component Time % Elap AvgEla
---------------------------------------- ----------- ------- ---------
CPU service 3934.97s 48.39% 0.000716
un-accounted for time 1363.01s 16.76%
db file sequential read 1122.00s 13.80% 0.032253
gc buffer busy 451.73s 5.56% 0.011746
log buffer space 451.64s 5.55% 0.123974
buffer busy waits 176.79s 2.17% 0.029579
gc cr block 2-way 156.49s 1.92% 0.003287
gc cr grant 2-way 100.20s 1.23% 0.006541
latch: cache buffers chains 98.92s 1.22% 0.005708
gc current grant 2-way 69.68s 0.86% 0.006728
latch: library cache 30.10s 0.37% 0.010030
row cache lock 28.95s 0.36% 0.018727
gc current block 2-way 26.72s 0.33% 0.003828
gc cr block busy 19.35s 0.24% 0.006802
gc current grant busy 15.30s 0.19% 0.004999
latch: row cache objects 14.28s 0.18% 0.006165
gc cr block 3-way 11.73s 0.14% 0.002952
gc current block 3-way 11.34s 0.14% 0.003440
log file sync 10.71s 0.13% 0.315066
enq: SQ - contention 9.14s 0.11% 0.060911
30. Timings for Single Block Reads – 3 node RAC
Single Block Read Times From AWR
0
5
10
15
20
25
30
35
40
45
500:00
1:40
3:20
5:00
6:40
8:20
10:00
11:40
13:20
15:00
16:40
18:20
20:00
21:40
23:20
Snapshot ID - 20 minutes interval
SingleBlockReadTimein
Milliseconds
Inst 1
31. Timings for Single Block Reads – 3 node RAC
Single Block Read Times From AWR
0
5
10
15
20
25
30
35
40
45
500:00
1:40
3:20
5:00
6:40
8:20
10:00
11:40
13:20
15:00
16:40
18:20
20:00
21:40
23:20
Snapshot ID - 20 minutes interval
SingleBlockReadTimein
Milliseconds
Inst 1
Inst 2
Inst3
32. The Facts About The I/O Bottleneck Problem
• Facts:
– 3 node RAC
– Same storage
– Single block read time for Instance 1 was substantially different from
read times for other instances during off hours
– 5 batch jobs during off hours
• The timings for Instance 1 are obviously not correct
• The sandwich syndrome (output from strace)
gettimeofday({1159440978, 931945}, NULL) = 0
pread(14, "6242003752302+254.000160054001050"..., 8192, 455
057408) = 8192
gettimeofday({1159440978, 944159}, NULL) = 0
• Waiting in runque for CPU exaggerates all wait times of the
process.