A lot of people use SQL in their everyday job, however there are some useful features, which are not so well known, but can save you a lot of time, code and processor power. The presentation is based on Oracle's SQL implementation but the features are relevant for all major relation DBMS.

2. Advanced SQL Queries
Stef CTO & Futurist

3. Disclaimer

5. DB Structure

6. batches batch_executions
id
name
id
batch_id
start_date
end_date
duration

7. Simple stuff

8. NVL

9. -- Execution time is
-- (end_date – start_date) if the batch is executed
-- (current time - start_date) if the batch is still
running

10. -- Execution time is
-- (end_date – start_date) if the batch is executed
-- (current time - start_date) if the batch is still
running
SELECT
(NVL(end_date, sysdate) – start_date) as duration
FROM batch_executions

11. DECODE

12. -- Fetch batch status
-- If end_date is not set, the batch is in progress
-- If end_date is set, the batch has finished

13. -- Fetch batch status
-- If end_date is not set, the batch is in progress
-- If end_date is set, the batch has finished
SELECT
DECODE(end_date, NULL, ‘IN PROGRESS’, ‘DONE’)
FROM batch_executions

14. Aggregation function
over
Aggregation function

15. -- Find the batch with longest average execution time

16. -- Find the batch with longest average execution time
SELECT MAX(avg_duration)
FROM (SELECT AVG(duration) as avg_duration
FROM batch_executions
GROUP BY batch_id)

17. -- Find the batch with longest average execution time
SELECT MAX(avg_duration)
FROM (SELECT AVG(duration) as avg_duration
FROM batch_executions
GROUP BY batch_id)

18. -- Find the batch with longest average execution time
SELECT MAX(avg_duration)
FROM (SELECT AVG(duration) as avg_duration
FROM batch_executions
GROUP BY batch_id)
SELECT MAX(AVG(duration))
FROM batch_executions
GROUP BY batch_id

19. WHERE’s my stuff

20. Handle missing parameters

21. WHERE (:name IS NULL
OR name LIKE :name )

22. IN

23. ANY, ALL

24. X > ANY (SELECT y FROM … )
X > ALL (SELECT y FROM … )

25. -- List batches with duration longer than the longest
average duration

26. -- List batches with duration longer than the longest
average duration
SELECT * FROM batch_executions
WHERE duration > ALL(SELECT AVG(duration)
FROM batch_executions
GROUP BY batch_id)

27. -- List batches with duration longer than the longest
average duration
SELECT * FROM batch_executions
WHERE duration > ALL(SELECT AVG(duration)
FROM batch_executions
GROUP BY batch_id)
SELECT * FROM batch_executions
WHERE duration > (SELECT MAX(AVG(duration))
FROM batch_executions
GROUP BY batch_id)

28. FROM stuff

29. UNION [ALL]

30. INTERSECT

31. -- Batches executed both today and yesterday

32. -- Batches executed both today and yesterday
SELECT batch_id FROM batch_executions
WHERE trunc(start_date) = trunc(sysdate - 1)
INTERSECT
SELECT batch_id FROM batch_executions
WHERE trunc(start_date) = trunc(sysdate)

33. MINUS

34. -- Batches which are never executed

35. -- Batches which are never executed
SELECT id FROM batches
MINUS
SELECT batch_id FROM batch_executions

36. GROUP BY stuff

37. ROLLUP

38. -- count of executions per day, month and year

39. -- count of executions per day, month and year
SELECT
to_char(start_date, 'YYYY') year,
to_char(start_date, 'mm') month,
COUNT(*)
FROM batch_executions
GROUP BY
to_char(start_date, 'YYYY'),
to_char(start_date, 'mm’)
UNION ALL
SELECT
to_char(start_date, 'YYYY') year,
NULL as month,
COUNT(*)
FROM batch_executions
GROUP BY
to_char(start_date, 'YYYY’)

40. -- count of executions per day, month and year
SELECT
to_char(start_date, 'YYYY') year,
to_char(start_date, 'mm') month,
to_char(start_date, 'dd') day,
COUNT(*)
FROM batch_executions
GROUP BY ROLLUP (
to_char(start_date, 'YYYY'),
to_char(start_date, 'mm'),
to_char(start_date, 'dd')
)

41. -- count of executions per day, month and year
SELECT
to_char(start_date, 'YYYY') year,
to_char(start_date, 'mm') month,
to_char(start_date, 'dd') day,
COUNT(*)
FROM batch_executions
GROUP BY ROLLUP (
to_char(start_date, 'YYYY'),
to_char(start_date, 'mm'),
to_char(start_date, 'dd')
)
GROUP BY
(year, month, day)
(year, month)
(year)

42. CUBE

43. -- count of executions per day, month and year
SELECT
to_char(start_date, 'YYYY') year,
to_char(start_date, 'mm') month
COUNT(*)
FROM batch_executions
GROUP BY CUBE (
to_char(start_date, 'YYYY'),
to_char(start_date, 'mm’)
)

44. -- count of executions per day, month and year
SELECT
to_char(start_date, 'YYYY') year,
to_char(start_date, 'mm') month
COUNT(*)
FROM batch_executions
GROUP BY CUBE (
to_char(start_date, 'YYYY'),
to_char(start_date, 'mm’)
)
GROUP BY
(year, month)
(year)
(month)

45. Analytic stuff
(Analytic Functions)

46. -- List batches and assigned them number based on
their start date
BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30 1
1 21-MAR-15 07:09:17 2
1 22-MAR-15 07:09:50 3
1 22-MAR-15 07:10:10 4
2 22-MAR-15 07:46:51 5
Expected

47. -- List batches and assigned them number based on
their start date
SELECT rownum, batch_id
FROM batch_executions
ORDER BY start_date

48. BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30 1
1 21-MAR-15 07:09:17 2
1 22-MAR-15 07:09:50 3
1 22-MAR-15 07:10:10 4
2 22-MAR-15 07:46:51 5
Expected
ActualBATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30 2
1 21-MAR-15 07:09:17 1
1 22-MAR-15 07:09:50 3
1 22-MAR-15 07:10:10 4
2 22-MAR-15 07:46:51 5

49. -- List batches and assigned them number based on
their start date
SELECT rownum, batch_id
FROM batch_executions
ORDER BY start_date
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)
WRONG
CORRECT

50. ROW_NUMBER

51. -- List batches and assigned them number based on
their start date
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)

52. -- List batches and assigned them number based on
their start date
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)
SELECT
row_number OVER (ORDER BY start_date),
start_date,
batch_id
FROM batch_executions

53. Function(arg1,..., argn) OVER ( [PARTITION BY <...>]
[ORDER BY <....>] [<window_clause>] )

54. Function(arg1,..., argn) OVER ( [PARTITION BY <...>]
[ORDER BY <....>] [<window_clause>] )

55. Function(arg1,..., argn) OVER ( [PARTITION BY <...>]
[ORDER BY <....>] [<window_clause>] )

56. -- List batches and assigned them number based on
their start date but only for the batch group
BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30 1
1 21-MAR-15 07:09:17 2
1 22-MAR-15 07:09:50 3
1 22-MAR-15 07:10:10 4
2 22-MAR-15 07:46:51 1
Expected

57. -- List batches and assigned them number based on
their start date but only for the batch group
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)

58. -- List batches and assigned them number based on
their start date but only for the batch group
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)
WRONG

59. -- List batches and assigned them number based on
their start date but only for the batch group
SELECT rownum, start_date, batch_id
FROM (SELECT start_date, batch_id
FROM batch_executions
ORDER BY start_date)
SELECT
row_number() OVER
(PARTITION BY batch_id ORDER BY
start_date),
start_date,
batch_id
FROM batch_executions
WRONG

60. BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30
1 21-MAR-15 07:09:17
1 22-MAR-15 07:09:50
1 22-MAR-15 07:10:10
2 22-MAR-15 07:46:51

61. BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30
1 21-MAR-15 07:09:17
1 22-MAR-15 07:09:50
1 22-MAR-15 07:10:10
2 22-MAR-15 07:46:51
BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30
1 21-MAR-15 07:09:17
1 22-MAR-15 07:09:50
1 22-MAR-15 07:10:10
2 22-MAR-15 07:46:51

62. BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30
1 21-MAR-15 07:09:17
1 22-MAR-15 07:09:50
1 22-MAR-15 07:10:10
2 22-MAR-15 07:46:51
BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30 1
1 21-MAR-15 07:09:17 2
1 22-MAR-15 07:09:50 3
1 22-MAR-15 07:10:10 4
2 22-MAR-15 07:46:51 1
BATCH_ID START_DATE NUM
1 12-DEC-14 07:09:30
1 21-MAR-15 07:09:17
1 22-MAR-15 07:09:50
1 22-MAR-15 07:10:10
2 22-MAR-15 07:46:51

63. -- List batches and assigned them number based on
their start date but only for the batch group
SELECT
row_number() OVER
(PARTITION BY batch_id ORDER BY
start_date),
start_date,
batch_id
FROM batch_executions

64. ROW_NUMBER | RANK | DENSE_RANK

65. SELECT
ROW_NUMBER() OVER (ORDER BY duration),
RANK() OVER (ORDER BY duration),
DENSE_RANK() OVER (ORDER BY duration),
DURATION
FROM batch_executions

66. SELECT
ROW_NUMBER() OVER (ORDER BY duration),
RANK() OVER (ORDER BY duration),
DENSE_RANK() OVER (ORDER BY duration),
DURATION
FROM batch_executions
RN RANK D_RANK DURATION
1 1 1 5
2 1 1 5
3 1 1 5
4 4 2 873

67. LEAD | LAG

68. -- List:
-- average duration and last run date for each batch
-- first previous and first following batch run dates
-- first previous and first following batch by average duration
SELECT
batch_id,
AVG(duration),
MAX(start_date),
LAG(batch_id) OVER (ORDER BY MAX(start_date)) prev_by_date,
LEAD(batch_id) OVER (ORDER BY MAX(start_date)) next_by_date,
LAG(batch_id) OVER (ORDER BY AVG(duration)) prev_by_duration,
LEAD(batch_id) OVER (ORDER BY AVG(duration)) next_by_duration
FROM batch_executions
GROUP BY batch_id

69. FIRST_VALUE | LAST_VALUE

70. MIN | MAX | COUNT | SUM | AVG

71. Window Clause

72. Function(arg1,..., argn) OVER ( [PARTITION BY <...>]
[ORDER BY <....>] [<window_clause>] )

73. ROW | RANGE

74. [ROW or RANGE] BETWEEN <start_expr> AND <end_expr>
<start_expr>
UNBOUNDED PRECEDING
CURRENT ROW
<sql_expr> PRECEDING or FOLLOWING.
<end_expr>
UNBOUNDED FOLLOWING or
CURRENT ROW or
<sql_expr> PRECEDING or FOLLOWING.

75. ROW vs RANGE

76. [ROW or RANGE] BETWEEN <start_expr> AND <end_expr>
<start_expr>
UNBOUNDED PECEDING
CURRENT ROW
<sql_expr> PRECEDING or FOLLOWING.
<end_expr>
UNBOUNDED FOLLOWING or
CURRENT ROW or
<sql_expr> PRECEDING or FOLLOWING.

77. Case Study

78. I2C Statistics

79. The Problem

80. The Problem
system status today

81. The Problem
system status today
system status by day

82. The Problem
system status today
system status by day
system status by month

83. Introducing
INV_TRANSITIONS
(inv, transition_date, old_status, new_status)

84. Data Aggregation

85. We define:
Initial State =
(available amount, financed amount, potential amount)
Transition change vector for each transition
e.g. Not Eligible -> Eligible (+500, 0, +500)

86. We define:
Initial State =
(available amount, financed amount, potential amount)
Transition change vector for each transition
e.g. Not Eligible -> Eligible (+500, 0, +500)
Note: We need only first and last day transitions

87. Example

88. The query

89. Thanks for watching

90. Questions

91. Resources
Analytic functions by Example:
http://www.orafaq.com/node/55
Oracle Base:
http://www.oracle-base.com/articles/misc/analytic-functions.php
http://www.oracle-base.com/articles/misc/lag-lead-analytic-
functions.php
http://www.oracle-base.com/articles/misc/first-value-and-last-value-
analytic-functions.php