This document discusses cost-based optimization and statistics in Firebird. It covers: 1) Rule-based optimization uses heuristics while cost-based optimization uses statistical data to estimate the cost of different access paths and choose the most efficient. 2) Statistics like selectivity, cardinality, and histograms help estimate costs by providing information on data distribution and amounts. 3) The optimizer aggregates costs from the bottom up and chooses the access path with the lowest total cost based on the statistical information.

1. Cost-based OptimizationandStatistics in Firebird Dmitry Yemanov The Firebird Project http://www.firebirdsql.org

3. Different queries and/or fetch strategies may benefit from different data access paths

4. Some information should exist in order to help the optimizer in guessing about the best access path

5. Optimization strategies

6. Rule-based (heuristics)

8. Indexed retrieval is better than a full table scan(and indexed loop join is better than a merge join)

9. B-tree has three levels of depth

10. Compound indices are better than simple ones

11. Drawbacks

12. Indices could be bad for some operations

13. User intentions are not taken into account

15. Every operation has an associated cost value

16. Cost value is calculated using statistical data

17. Cost is aggregated from bottom up in the access path

18. Drawbacks

19. Complex implementation

20. Slow optimization process

22. Represents a fraction of rows from a row set

23. Lies in the value range 0.0 to 1.0

24. Cardinality

25. Represents number of rows in a row set

27. Is a function of the estimated cardinalities

28. Represents computational complexity of the retrieval

29. Measurement

30. Cost value linearly depends on the number of logical reads required to perform an operation

31. Logical read is equal to a single page fetch

33. cost = base cardinality

34. Unique index scan

35. cost = b-tree level + 1

36. Range index scan

39. Stored in a database or estimated at runtime

41. Small tables:number of used record slots on the data pages

42. Large tables:number of used data pages / average record length

44. 1 / number of distinct keys in the index

45. Maintained per segment: (A), (A, B), (A, B, C)

46. Assumes uniform distribution of values

47. Calculated during index creation or upon request(SET STATISTICS statement)

48. Stored on the index root page

50. Selectivity close to 1.0 suggests a full scan

51. What Indices to Use

52. Compare index selectivities and index scan costs

53. Consider segment operations for compound indices

54. Calculate selectivities for AND and OR operations

55. Order of Streams in Loop Joins

57. Average page fill factor

58. Average row length(both help with a better base cardinality estimation)

60. B-tree depth

61. Average key length(both help with a better cost estimation for index scans)

64. Selectivity(core feature, required to estimate costs)

65. Number of NULLs(useful for selectivity estimations for IS [NOT] NULL)

68. Selectivity of columns being grouped by

69. Loop Join vs Merge Join vs Hash Join

70. Cardinality of tables and filtering predicates

71. Index Usage

72. Number of NULLs or histogram

73. Index Navigation vs External Sorting