postgresql sql-1992 sql-2003

1. MODERN SQL
SQL Beyond 1992

2. Why use an RDBMS?

3. Things to look at when choosing a database
blah blah blah
• Consistency, availability, and partition tolerance (CAP)
• Robustness and reliability
• Scalability
• Performance and speed
• Operational and querying capabilities
• Database integrity and constraints
• Database security
• Database vendor/system funding, stability, community, and level of establishment
• Talent pool and availability of relevant skills
• The type and structure of data being stored, and the ideal method of modeling the data
•In other words – IT DEPENDS
• And don’t let any database employee tell you otherwise 

4. The SQL you use…
■ Is from 1992
– At least that’s a step up from SQL-89
– Until MSVC 2015 C was stuck on C89
■ The first SQL standard was created in 1986
– COBOL, FORTRAN, Pascal and PL/I
– I now feel old
■ Was driven by large companies
– See if you can name some

5. We are ANSI –
and we like
standards!
“
Second, because there was no outside “buyer” to
shape the content of the Core level of SQL99, it
was enlarged to such an extent that to
implement it all is close to impossible for all
vendors except for two or three. In short, the size
of core is a natural barrier to practical product
development.
”
Michael Gorman
Secretary of the ANSI Database Languages Committee
http://tdan.com/is-sql-a-real-standard-
anymore/4923
SQL-92
SQL-99
SQL-2003
SQL-2011

6. Reporting Reporting Reporting
■ Almost all the features we’ll discuss are most useful for reporting
■ Some are syntactic sugar (or run faster in cases) than traditional SQL
■ Mysql doesn’t have any of this (sorry folks) – maybe WITH in 8.0
– https://dveeden.github.io/modern-sql-in-mysql/
– Go here and +1 all the feature requests!
■ PostgreSQL has it all, plus some other goodies
– But use a new version, I recommend 9.5

7. But it’s 2016!!

8. OLAP
Grouping sets, CUBE, and ROLLUP

9. OLAP
■ Online analytical processing
■ Processes multi-dimensional analytical (MDA) queries swiftly
■ Consolidation (roll-up)
– aggregation of data that can be accumulated and computed in one or more
dimensions
■ Drill-down
– navigate through the details
■ Slicing and dicing
– take out (slicing) a specific set of data and view (dicing) the slices from different
dimensions

10. GROUPING SETS, ROLLUP, CUBE
■ GROUPING SETS ( ( e1, e2, e3, ... ), ( e1, e2 ), ( ))
– allow you to choose how you do sets of columns that you group by
■ ROLLUP ( e1, e2, e3, ... )
– is shorthand the given list of expressions and all prefixes of the list including the
empty list, useful for hierarchical data
■ CUBE ( e1, e2, ... )
– is shorthand for the given list and all of its possible subsets

11. Getting some
information
grouped multiple
ways…

12. GROUPING SET example
■ https://gist.github.com/auroraeosrose/b6b71780ba4c91cd02e6d175a6eeb49a

13. ROLLUP example

14. Why do we care?
■ Simplify queries
■ Perform fewer queries, get more data out of the same query
■ Group information on multiple (and complex) dimensions

15. Support
■ MySQL (and MariaDB) have GROUP BY …WITH ROLLUP
– That’s it, just rollup
– And it’s kind of broken syntax too compared to other DBs
■ PostgreSQL was late to the party (9.5) but implemented ALLTHETHINGS
■ SQL Server, Oracle, DB2 have had this stuff for ages (plus a bunch of proprietary olap
features in addition!)
■ SQLite IS missing this stuff (bad SQLite, bad!)

16. WITH (RECURSIVE)
CTE for you and for me

17. Subqueries suck to read

18. WITH – organize complex queries
■ WITH query_name (column_name1, ...)AS
(SELECT ...)
SELECT ...
■ A way to organize queries
■ Also called common table expression (CTE) and sub-query factoring
■ Makes highly complex queries sane to read and understand

19. WITH example
■ https://gist.github.com/auroraeosrose/82cc6c420d7749336ef474c87df50841

20. Why do we care?
■ If it’s easier to read it’s easier to maintain
■ Assign column names to tables
■ Hide tables for testing (with names HIDETABLES of the same name)

21. WITH RECURSIVE
■ The optional RECURSIVE modifier changesWITH from a mere syntactic convenience
into a feature that accomplishes things not otherwise possible in standard SQL. Using
RECURSIVE, aWITH query can refer to its own output
■ WAT?

22. With recursive example

23. TOTHEVM!
■ Ubuntu 16.04
■ sudo apt-get install postgresql
■ sudo –i –u postgres
■ psql

24. Why do we care?
■ Row generators
– Fake data
– Series
– Dates
■ Processing Graphs
■ Finding distinct values
– Loose index scan

25. Support
■ Basically everyone but MySQL
– Yes, even SQLite 
■ Only PostgreSQL REQUIRES the recursive keyword
– They do some weird things withWITH

26. LATERAL
The foreach loop of SQL

27. LATERAL – joining by foreach
■ a LATERAL join is like a SQL foreach loop, in which the db will iterate over each row in
a result set and evaluate a subquery using that row as a parameter.
■ A lateral join can reference other tables in the query!
■ Generally lateral joins are faster (the optimizer gets to have fun)

28. Lateral join example
■ https://gist.github.com/auroraeosrose/73fb8d0779ef4c0251754f38eea228de

29. TOTHEVM!

30. Support
■ Everyone but… MySQL and SQL Server
– Well, except in SQL Server you can use cross/outer APPLY to do the same thing

31. 2003 has arrived…

32. FILTER
And the CASE fakeout

33. FILTER – selective aggregates
■ SUM(<expression>) FILTER(WHERE <condition>)
■ Works on any aggregate function
– Including array_agg
■ Faking it –
– SUM(CASEWHEN <condition>THEN <expression> END)
– COUNT(CASEWHEN <condition>THEN 1 END)

34. Filter example
■ https://gist.github.com/auroraeosrose/f649e997ffd94df57c827ab3b0ee7d1b

35. CASE instead

36. TOTHEVM!
■ Which is faster – case or filter?
■ Why is this not the #1 request everywhere
– Look, we’ll give you a keyword that optimizes the crap out of your query and you
can do more with just 1 query

37. Support
■ Only PostgreSQL has it 
■ You can use CASE on almost any other DB to fake it

38. WINDOW FUNCTIONS
Rank, Over, Partition By

39. Window Functions
■ Define which rows are visible at each row
■ OVER() makes all rows visible at each row
■ OVER(PARTITION BY) segregates like a GROUP BY
■ OVER(ORDER BY … BETWEEN) segregates using < >

40. As a query writer I want to
■ Merge rows that have the same things
– GROUP BY
– DISTINCT
■ Aggregate data from related rows
– Requires a GROUP BY
– Uses aggregate functions
■ BUTATTHE SAMETIME

41. OVER (PARTION BY)

42. TOTHEVM!
■ You can do a LOT more withWindowing
■ You can page
■ You can do ranges and between
■ You can window more than once
■ I could do a whole talk on windowing!

43. OVER (ORDER BY …)

44. Windowing Functions
row_number() number of the current row within its partition, counting from 1
rank() rank of the current row with gaps
dense_rank() rank of the current row without gaps
percent_rank() relative rank of the current row: (rank - 1) / (total rows - 1)
cume_dist()
relative rank of the current row: (number of rows preceding or peer
with current row) / (total rows)
ntile(num_buckets integer)
integer ranging from 1 to the argument value, dividing the partition
as equally as possible
lag(value anyelement)
returns value evaluated at the row that is offset rows before the
current row within the partition
lead(value anyelement)
returns value evaluated at the row that is offset rows after the
current row within the partition
first_value(value any) returns value evaluated at the row that is the first row
last_value(value any) returns value evaluated at the row that is the last row
nth_value(value any, nth integer)
returns value evaluated at the row that is the nth row of the
window frame (counting from 1); null if no such row

45. Support
■ PostgreSQL
■ SQL Server
■ Oracle
■ DB2
– Both SQLite and MySQL are missing support

46. To the future!

47. NOSQL INYOUR SQL
jsonb and PostgreSQL

48. But I like nosql! But I need SQL…

49. Json types in Postgresql
json
■ String internal representation
■ http://rfc7159.net/rfc7159
– previously supported
http://www.ietf.org/rfc/rfc4627.t
xt
■ Stores exact text, reparsed on each
execution
jsonb
■ Binary internal representation
■ Can have indexes on stuff inside
■ Has “shadow types”
■ http://rfc7159.net/rfc7159
■ De-duplicates and decomposes to
binary format

50. Creation

51. Simple Selection

52. TOTHEVM!
■ But wait, there’s more

53. Operators
Operator Example Example Result
->
'[{"a":"foo"},{"b":"bar"},{"c":"baz"}]'::j
sonb->2
{"c":"baz"}
-> '{"a": {"b":"foo"}}'::jsonb->'a' {"b":"foo"}
->> '[1,2,3]'::jsonb->>2 3
->> '{"a":1,"b":2}'::jsonb->>'b' 2
#>
'{"a": {"b":{"c":
"foo"}}}'::jsonb#>'{a,b}'
{"c": "foo"}
#>>
'{"a":[1,2,3],"b":[4,5,6]}'::jsonb#>>'{a
,2}'
3

54. Moar Operators
Operator Example
@> '{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb
<@ '{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb
? '{"a":1, "b":2}'::jsonb ? 'b'
?| '{"a":1, "b":2, "c":3}'::jsonb ?| array['b', 'c']
?& '["a", "b"]'::jsonb ?& array['a', 'b']
|| '["a", "b"]'::jsonb || '["c", "d"]'::jsonb
- '{"a": "b"}'::jsonb - 'a'
- '["a", "b"]'::jsonb - 1
#- '["a", {"b":1}]'::jsonb #- '{1,b}'

55. Acknowledgements and Resources
■ MarkusWinand
– http://modern-sql.com
■ Daniël van Eeden
– https://dveeden.github.io/modern-sql-in-mysql
■ PostgreSQL
– https://www.postgresql.org/docs/9.5

56. About Me
 http://emsmith.net
 auroraeosrose@gmail.com
 twitter - @auroraeosrose
 IRC – freenode – auroraeosrose
 #phpmentoring
 https://joind.in/talk/d5ba5