So you are young and single – you are standing in a crowded room and you are looking to connect with all the people that match the criteria of your perfect partner. How do you identify and grab the details of those individuals as quickly as possible, without spending too much time talking to people you will never get on with. This presentation will use a situation we have all been familiar with to explain how the optimizer works, the steps in the optimisation process and some of the rules that help it to score a match as efficiently as possible.

1. www.sagecomputing.com.au
penny@sagecomputing.com.au
OHarmony – Finding Your Perfect Match
How the Optimizer Works
Penny Cookson
SAGE Computing Services
SAGE Computing Services
Customised Oracle Training Workshops and Consulting

2. WARNING
This presentation contains material which is not politically correct
Includes adult concepts
May contain strong language

3. SAGE Computing Services
Customised Oracle Training Workshops and Consulting
Penny Cookson
Managing Director and Principal Consultant
Working with Oracle products since 1987
Oracle Magazine Educator of the Year 2004
www.sagecomputing.com.au
penny@sagecomputing.com.au

4. Optimiser Step 1 - Transformation
Query rewrite for
materialised views
Transitive conditions
OR expansion
View merging
Predicate pushing
Join factorisation

5. COL1 = :B1
AND COL2 >= :B2 AND COL2 < :B3
AND COL3 >= :B4
AND COL4 = :B5
…………………………………….
Col1 Col2 Col3 Col4 Col5 Col6
How many of these
are there likely to
be?
How many rows will be returned?
There are 23 million rows in this table

6. ATTRIBUTE1 = :B1
AND ATTRIBUTE2 >= :B2
AND ATTRIBUTE2 < :B3
AND ATTRIBUTE3 >= :B4
AND ATTRIBUTE4 = :B5
How many
of these are
there likely
to be?
Looking for your perfect match
There are 11 million males
in Australia

7. MARRIED = ‘N’
AND AGE >=25 AND AGE <30
AND HEIGHT >= 6ft 2 in
AND JOB=‘DBA’
How many
of these are
there likely
to be?
There are 11 million males
in Australia

8. The attribute Married
has two distinct
values Yes or No
50% 50%
We assume 50% of
each
How many people
satisfy the criteria
Married = ‘N’?
There are 11 million
males in Australia
Number of Unmarried males is
11,000,000/2 = 5,500,000

9. How many people satisfy the criteria Married = ‘N’?

10. More Statistics
6 in every 10 males are
married
So - Number of Unmarried males
is 4,400,000

11. This is what we did originally
begin
dbms_stats.gather_schema_stats
(ownname=>'AUSOUG',
method_opt => 'for all columns size 1' );
end;
How can Oracle get better statistics?

12. begin
dbms_stats.gather_table_stats
(ownname=>'AUSOUG',
tabname=>'MEN',
method_opt => 'for all columns size 1,
for columns size auto married' );
end;
Create a histogram only for Married
This is no longer relevant Because we have this

13. Cummulative

14. http://jonathanlewis.wordpress.com/2010/10/05/frequency-histogram-4/

15. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE married = 'N‘;
SELECT dbms_xplan.display_cursor
('5zyycq4y22j9g',format=>'ALLSTATS LAST')
FROM dual;
Now it gets it right

16. MARRIED = ‘N’
AND AGE >=25 AND AGE <30
AND HEIGHT >= 6ft 2 in
AND JOB=‘DBA’
How many
of these are
there likely
to be?
There are 11 million males
in Australia
MARRIED = ‘N’  40% 

17. Age statistics
8.2% of men are BETWEEN
25 and 29

18. Age statistics

19. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE age >=25 and age < 30;

20. create or replace function raw_to_num(i_raw raw)
return number as m_n number;
begin
dbms_stats.convert_raw_value(i_raw,m_n);
return m_n;
end;
/
create or replace function raw_to_date(i_raw raw)
return date as m_n date;
begin
dbms_stats.convert_raw_value(i_raw,m_n);
return m_n;
end;
/
create or replace function raw_to_varchar2(i_raw raw)
return varchar2 as m_n varchar2(20);
begin
dbms_stats.convert_raw_value(i_raw,m_n);
return m_n;
end;
/
http://jonathanlewis.wordpress.com/2006/11/29/low_value-high_value/

21. SELECT
column_name,
decode(data_type,
'VARCHAR2',to_char(raw_to_varchar2(low_value)),
'DATE',to_char(raw_to_date(low_value)),
'NUMBER',to_char(round(raw_to_num(low_value),2))
) low_value,
decode(data_type,
'VARCHAR2',to_char(raw_to_varchar2(high_value)),
'DATE',to_char(raw_to_date(high_value)),
'NUMBER',to_char(round(raw_to_num(high_value),2))
) high_value
FROM user_tab_columns
WHERE table_name='MEN';
http://jonathanlewis.wordpress.com/2006/11/29/low_value-high_value/

23. 5.10204… % * 11,000,000 = 561,224
0 98
25 30
SELECT TRUNC(11000000*(30-25)/(98)),trunc((30-25)/(98)*100,6) from dual

24. Add a histogram
begin
dbms_stats.gather_table_stats(ownname=>'AUSOUG',
tabname=>'MEN',
method_opt =>
'for all columns size 1 for columns size auto married, for columns size
254 age ' );
end;

25. SELECT
trunc(h.endpoint_value,2)
,h.endpoint_repeat_count
from user_tab_histograms h, user_tables t
WHERE t.table_name = h.table_name
AND h.table_name = 'MEN'
AND h.column_name = 'AGE'
ORDER BY endpoint_value

26. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE age >=25 and age < 30;
SELECT
dbms_xplan.display_cursor('6aub3fn962277',format=>'ALLSTATS LAST')
FROM dual
Not perfect but better

27. MARRIED = ‘N’
AND AGE >=25 AND AGE <30
AND HEIGHT >= 6ft 2 in (188cm)
AND JOB=‘DBA’
How many
of these are
there likely
to be?
There are 11 million males
in Australia
MARRIED = ‘N’  40%
AND AGE >=25 AND <30  8.2%



28. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE height >= 188
SELECT dbms_xplan.display_cursor('bf4wbtgb9zptq',
format=>'ALLSTATS LAST')
FROM dual
How many men are >= 188cm (we have gathered a histogram)

29. MARRIED = ‘N’
AND AGE >=25 AND AGE <30
AND HEIGHT >= 6ft 2 in (188cm)
AND JOB=‘DBA’
How many
of these are
there likely
to be?
There are 11 million males
in Australia
MARRIED = ‘N’  40%
AND AGE >=25 AND <30  8.2%
AND HEIGHT >= 6ft 2 in (188cm)  2.9%




30. 11000000*40/100 *8.2/100 * 2.9/100 =10,463
MARRIED = ‘N’  40%
AND AGE >=25 AND <30  8.2%
AND HEIGHT >= 6ft 2 in (188cm)  2.9%
SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE married = 'N'
AND age >=25 AND age < 30
AND height >= 188

32. Height Statistics

33. 5.3% of males are >= 6ft 2inches
Are these
statistics out of
date?
- do select of
last_analyzed

34. 5.3% of males are >= 6ft 2inches
Note the
correlation
between
gender, age and
height

35. SELECT c.column_name, t.num_rows "Number of Rows",
c.num_distinct "Distinct Values",
c.histogram "Histogram"
FROM user_tables t, user_tab_col_statistics c
WHERE t.table_name = c.table_name
AND t.table_name = 'MEN'

36. SELECT /*+ GATHER_PLAN_STATISTICS */
COUNT(*)
FROM men
WHERE age =6
AND height = 174

37. BEGIN
dbms_stats.gather_table_stats(ownname=>'AUSOUG',
tabname=>'MEN', estimate_percent=>NULL,
method_opt =>
'for all columns size 1 for columns size auto married height, for
columns size 254 age, for columns (age, height) size 2048 ' );
END;
Gather extended statistics

38. SELECT *
FROM dba_stat_extensions
WHERE table_name = 'MEN'

39. SELECT c.column_name, t.num_rows "Number of Rows",
c.num_distinct "Distinct Values",
c.histogram "Histogram"
FROM user_tables t, user_tab_col_statistics c
WHERE t.table_name = c.table_name
AND t.table_name = 'MEN'

40. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE age =6
AND height = 174

41. ALTER SESSION SET EVENTS ' 10053 trace name context forever ‘;
EXPLAIN PLAN FOR …………;
ALTER SESSION SET EVENTS ' 10053 trace name context off ‘;

42. SELECT /*+ GATHER_PLAN_STATISTICS */
COUNT(*)
FROM men
WHERE age = 30
AND height = 174

43. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*)
FROM men
WHERE married = 'N'
AND age >=25 AND age < 30
AND height >= 188
When we combine range checks it gets it wrong

44. 10053 trace file – its not looking at the extended stats
ALTER SESSION SET EVENTS ' 10053 trace name context forever ‘;
EXPLAIN PLAN FOR …………;
ALTER SESSION SET EVENTS ' 10053 trace name context off ‘;

45. BEGIN
DBMS_STATS.DROP_EXTENDED_STATS('AUSOUG', 'MEN',
'("AGE","HEIGHT","MARRIED")') ;
END;
BEGIN
dbms_stats.purge_stats(sysdate);
END;

46. What about Statistics Feedback?

48. Execute again

50. begin
dbms_spd.flush_sql_plan_directive;
end;
Clear any existing SQL Plan Directives
SELECT d.directive_id, d.type, d.state, d.reason, d.created,
o.object_name, o.subobject_name, o.notes, o.owner
FROM dba_sql_plan_directives d, dba_sql_plan_dir_objects o
WHERE o.directive_id = d.directive_id
AND o.owner NOT IN ('XDB','SYS','SYSTEM')
ORDER BY directive_id desc;
begin
dbms_spd.drop_sql_plan_directive(3579438123315094543);
end;

51. ALTER SYSTEM FLUSH shared_pool;
ALTER SESSION SET statistics_level = ALL;
Clear any existing plans and gather plan
statistics

52. Execute each of these twice

53. Only the last two use
statistics feedback

55. UNMARRIED = ‘Y’
AND AGE >=25 AND AGE <30
AND HEIGHT >= 6ft 2 in (188cm)
AND JOB=‘DBA’
How many
of these are
there likely
to be?
There are 11 million males
in Australia
UNMARRIED = ‘Y’  40%
AND AGE >=25 AND <30  8.2%
AND HEIGHT >= 6ft 2 in (188cm)  2.9%
We have no idea what percentage of males are DBAs




56. begin
dbms_stats.delete_schema_stats(ownname=>'AUSOUG' );
end;
SELECT COUNT(*)
FROM men
WHERE job = 'DBA'
Actual value is 1,000,014

57. Now we know how many matches to expect
what is the best way to get to them

58. Two main types:
Pretty cruisy really – almost anyone will do
Really very picky – he must be just right

60. The Oracle approach to
Pretty cruisy really – almost anyone will do
SELECT COUNT(*)
FROM men
WHERE age >=20 AND age < 50
AND married = 'N'
AND job != 'LAWYER'

61. Full table scan
WHERE col1 = ' bbbbb '
aaaaa
bbbbb
ccccc
ddddd
eeeee
aaaaa
ggggg
ccccc
ddddd
eeeee
aaaaa
kkkkk
ccccc
ddddd
eeeee
aaaaa
bbbbb
ccccc
ddddd
eeeee
aaaaa
bbbbb
bbbbb
ddddd
eeeee
Multi
block
read

62. SELECT COUNT(*)
FROM men
WHERE age >=20 AND age < 50
AND married = 'N'
AND job != 'LAWYER'

63. Really very picky – he must be just right
OHarmony
I tell you what I want and you just give me
the phone numbers where I can contact
them

64. OHarmony

65. OHarmony
Brown
DBA
188
120,000
N
25Age From Age To 30
IQ (min)
Married
Smoker
Eyes
Height From
Salary (min annual)
120
N
195Height To
Preferred Job Type

66. OHarmony
Brown
DBA
188
120,000
N
25Age From Age To 30
IQ (min)
Married
Smoker
Eyes
Height From
Salary (min annual)
120
N
195Height To
Preferred Job Type
John Smith 9999 9999

68. Index range scan
a
z
bbbb rowid
bbbb rowid
aaaaa
bbbbb
ccccc
ddddd
eeeee
aaaaa
ggggg
ccccc
ddddd
eeeee
aaaaa
kkkkk
ccccc
ddddd
eeeee
aaaaa
bbbbb
ccccc
ddddd
eeeee
WHERE
col1 = ' bbbbb '

69. SELECT id, surname, firstname
FROM men
WHERE iq = 155;

70. SELECT COUNT(surname)
FROM men
WHERE iq > 154;
SELECT COUNT(surname)
FROM men
WHERE iq > 153;
9 rows
0.00008%
137,488 rows
1.25%

71. OHarmony
DEVELOPER
195
120,000
N
25Age From Age To 30
IQ (min)
Married
Smoker
Eyes
Height From
Salary (min annual)
N
Height To
Preferred Job Type

72. Oharmony has
provided
14 matches in
14 locations
Poorly clustered

73. Oharmony has
provided
14 matches in 2
locations
Well clustered

74. SELECT i.index_name, i.distinct_keys, i.num_rows,
i.clustering_factor, t.blocks
FROM user_indexes i, user_tables t
WHERE t.table_name = i.table_name
AND t.table_name = 'MEN';

77. CREATE TABLE men2
AS SELECT * FROM men
ORDER BY IQ;
CREATE INDEX MEN2_IQ_N4 ON men2(iq);
begin
dbms_stats.gather_table_stats(ownname=>'AUSOUG',
tabname=>'MEN2', estimate_percent => null,
cascade=>true,
method_opt =>
'for all columns size 1 for columns size 2000 iq ' );
end;

78. SELECT i.distinct_keys, i.num_rows,
i.clustering_factor, t.blocks
FROM user_indexes i, user_tables t
WHERE t.table_name = i.table_name
AND i.index_name = 'MEN2_IQ_N4';
previously

79. previously

80. Optimizer Mode

83. Optimizer Mode
All_Rows
First_Rows
Tends towards:

84. SELECT id, surname, firstname
FROM men
WHERE married = 'N'
AND age_range = '25-29'
AND height = 188
AND iq = 120
AND job = 'DBA';
Bitmap indexes

85. With Bitmap indexes

86. Sorting
With B*Tree indexes
SELECT id, surname, firstname
FROM men
WHERE married = 'N'
AND age_range = '25-29‘
AND height = 188
AND iq = 120
AND job= 'DBA';

87. access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
Identify each join path
How many rows am I likely to get?
What is the best join method?
All I have talked about so far is Access to one table
– how does it JOIN them together ?

88. access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
1
2
3

89. access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
1
2 3

90. access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
access conditions
number of rows?
access method?
1
2 3
for each join path
for each join – what is the best JOIN METHOD?

91. Joins methods – Nested Loop with index
1 2
A.COL1 = B.COL2
A B
COL1 = 1
B.COL2
index
ACCESS
ROWS
WHERE
COL2 = 1
COL2 = 1
COL2 = 1
COL2 = 1

93. Joins methods – Hash join
A
B
HASH
TABLE1

95. Joins methods – cartesian
A
B

97. SELECT COUNT(*)
FROM men
WHERE married = 'N'
AND age < 12
AND height >= 188
The wrong plan 0 rows

98. 0 rows
ALTER SESSION SET OPTIMIZER_INDEX_COST_ADJ = 5;
SELECT COUNT(*)
FROM men
WHERE married = 'N'
AND age < 12
AND height >= 188
This is even worse

99. SELECT /*+ INDEX_COMBINE(MEN MEN_AGE_N1,MEN_HEIGHT_N1) */ COUNT(*)
FROM men
WHERE married = 'N'
AND age < 12
AND height >= 188
0 rowsBetter

100. SELECT COUNT(surname)
FROM men
WHERE height between 159.4 AND 160
26188 rows
SELECT /*+ FULL(MEN) */ COUNT(surname)
FROM men
WHERE height between 159.4 AND 160

101. 10,299,997 rowsSELECT COUNT(surname), COUNT( start_date)
FROM men m, events e
WHERE e.men_id (+) = m.id
AND m.iq = 156
SELECT /*+ USE_HASH(M,E) */ COUNT(surname), COUNT( start_date)
FROM men m, events e
WHERE e.men_id (+) = m.id
AND m.iq = 156

102. www.sagecomputing.com.au
penny@sagecomputing.com.au
Questions?
Penny Cookson
SAGE Computing Services
SAGE Computing Services
Customised Oracle Training Workshops and Consulting