1. Harder Better
Faster Stronger
PostgreSQL 9.1
Selena Deckelmann
PostgreSQL Global Development Group
So
O mS
eCC
O on
N
fer
20
en
11
e c

2. 11
20
N
O
SC
O

3. Say “Postgres.”
c e
11
en
20
fer
O on
N
eCC
O mS
So

4. c e
11
en
20
fer
O on
N
eCC
O mS
So

5. “World’s most advanced open source database.”
So
O mS
eCC
O on
N
fer
20
en
11
e c

6. c e
11
en
20
fer
O on
N
eCC
O mS
So

7. c e
11
en
20
fer
O on
N
eCC
O mS
So

8. c e
11
en
20
fer
O on
N
eCC
O mS
So

9. Postgres disrupts the
commercial relational
database industry.
So
O mS
eCC
O on
N
fer
20
en
11
e c

10. Disruptive forces
Licensing
“Enterprise” Web development
Proprietary DBA Career Path
So
O mS
eCC
O on
N
fer
20
en
11
e c

11. Licensing
Drop-in replacement for custom application
development.
Cheap Data Warehousing
Ecosystem!
So
O mS
eCC
O on
N
fer
20
en
11
e c

12. “Enterprise”
Web development
Postgres is a “Developers database” -
Extension is encouraged!
Database of choice for Django
Oracle wants PHP developers
So
O mS
eCC
O on
N
fer
20
en
11
e c

13. “Visit the Ruby and Python Topic
for more information pertaining
to Ruby/RoR, Python and other
non-PHP scripting languages.”
So
O mS
eCC
O on
N
fer
20
en
11
e c

14. DBA Career path
Companies can’t hire Postgres DBAs fast
enough.
Postgres is close enough to Oracle, that
Oracle DBAs can jump in.
What we do better: Data loading,
automation, TUI and GUI admin tools
So
O mS
eCC
O on
N
fer
20
en
11
e c

15. And... Stability
Support major releases for at least 5 years.
Commits to HEAD are stable.
Bugs reported are fixed in <48 hrs.
So
O mS
eCC
O on
N
fer
20
en
11
e c

16. 10
features
Harder Better
More secure & robust: We made us better:
SE Postgres Writeable CTE
Replication Tools Per-column collations
Extensions
Faster Stronger
Performance: First in class:
KNN Indexes Serializable Snapshot Isolation
Unlogged Tables Foreign Data Wrappers
So
O mS
e
Synchronous Replication
CC
O on
N
fer
20
en
11
e c

17. Harder
c e
11
en
20
fer
O on
N
eCC
O mS
So

18. Harder Replication
Tools
WAL-based replication:
File-based (default 16-MB files)
Streaming (per-WAL record) (9.0)
Asynchronous
Warm standby
Hot standby (9.0)
Synchronous (9.1)
new!
Cascading replication (just committed... 9.2)
new!
http://www.pgcon.org/2010/schedule/attachments/149_PGCon2010-Built-in-Replication.pdf
So
O mS
eCC
O on
N
fer
20
en
11
e c

19. Harder Replication
Tools
pg_stat_replication view
shows all replicas and their statuses
pg_basebackup
single-command cloning over port 5432
http://www.postgresql.org/docs/9.1/static/app-pgbasebackup.html
promote to master
pg_ctl promote
formerly only could do this by creating a trigger file
new functions
pg_last_xact_replay_timestamp()
pg_xlog_replay_pause()
So
O mS
eCC
O on
N
fer
20
en
11
e c

20. Harder
SE-Postgres
SE-Linux security label support!
Originally a NSA-funded Linux kernel extension
For tablespace, schema, table, column,
sequence, view and procedure object classes
Will have integrated support in pgAdmin!
http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-1.html
So
O mS
http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-2.html
eCC
O on
N
fer
20
http://wiki.postgresql.org/wiki/SEPostgreSQL_Documentation
en
11
e c

21. Better
Writeable CTE
But first - What’s a CTE?
Common Table Expressions
A temporary table or VIEW that exists for just a
single query.
http://developer.postgresql.org/pgdocs/postgres/queries-with.html#QUERIES-WITH-
MODIFYING
So
O mS
eCC
O on
N
fer
20
en
11
e c

22. Better
c e
11
en
20
fer
O on
N
eCC
O mS
So

23. Better Common Table
Expression Example
WITH RECURSIVE t1(a, b) AS (
VALUES(0,0)
UNION ALL
SELECT CASE CAST(b AS BOOLEAN)
WHEN b % 3 = 0 THEN b
WHEN b % 5 = 0 THEN b
END,
b+1 Euler Project, question 1:
FROM t1 If we list all the natural numbers
WHERE b < 1000 below 10 that are multiples of 3 or
) 5, we get 3, 5, 6 and 9. The sum of
SELECT sum(a) FROM t1; these multiples is 23.
Find the sum of all the multiples
of 3 or 5 below 1000.
So
O mS
eCC
O on
N
fer
20
http://wiki.postgresql.org/wiki/Euler_Project,_Question_1
en
11
e c

24. Better
Writeable CTE
Makes maintaining partitions easier:
WITH deleted_posts AS (
DELETE FROM posts
WHERE created < now()
- '6 months'::INTERVAL
RETURNING *
)
SELECT user_id, count(*) FROM deleted_posts group BY 1;
http://www.depesz.com/index.php/2011/03/16/waiting-for-9-1-writable-cte/
So
O mS
eCC
O
http://xzilla.net/blog/2011/Mar/Upserting-via-Writeable-CTE.html
on
N
fer
20
en
11
e c

25. Better Per-column
collation
Supports multiple languages in the same table, and have ORDER BY
sort by language order (assuming you’ve loaded the correct collation
library)
BEFORE:
$ select * from normal_polish order by some_text;
some_text
-----------
alfa
car
ćma
coś
lama
So
O mS
eCC
O on
N
fer
20
en
11
e c

26. Better Per-column
collation
AFTER:
$ select * from normal_polish order by some_text;
some_text
-----------
alfa
car
coś
ćma
lama
So
O mS
eCC
O on
N
fer
20
en
11
e c

27. Better
Extensions
Postgres-specific package management for
contrib/, PgFoundry projects, tools.
Like Oracle "packages"
End user view: http://facility9.com/2011/03/postgresql-extensions/
Docs: http://developer.postgresql.org/pgdocs/postgres/sql-
createextension.html
So
O mS
eCC
O on
N
fer
20
en
11
e c

28. Extensions
Better 11
en
c e
20
fer
O on
N
eCC
O mS
So

29. Faster
c e
11
en
20
fer
O on
N
eCC
O mS
So

30. Faster K-Nearest
Neighbor Indexes
Incredibly fast search for things that are
"near" a particular object
Operator is: <->
Datatypes supported: pg_trgm contrib and
btree_gist datatypes (point)
Wiki: http://wiki.postgresql.org/wiki/
What's_new_in_PostgreSQL_9.1#K-Nearest-Neighbor_Indexing
Developer: http://www.sai.msu.su/~megera/postgres/talks/
pgday-2010.pdf
So
O mS
eCC
O on
N
fer
20
en
11
e c

31. Faster K-Nearest
Neighbor Indexes
$ create table test ( position point );
$ insert into test (position) select point( random() * 1000, random() *
1000) from generate_series(1,1000000);
$ create index q on test using gist ( position );
$ select *, position <-> point(500,500) from test order by position <->
point(500,500) limit 10;
$ explain analyze select *, position <-> point(500,500) FROM test
ORDER BY position <-> point(500,500) limit 10;
QUERY PLAN
Limit (cost=0.00..0.77 rows=10 width=16) (actual time=0.164..0.475
rows=10 loops=1)
-> Index Scan using q on test (cost=0.00..76512.60 rows=1000000
width=16) (actual time=0.163..0.473 rows=10 loops=1)
So
O
Order By: ("position" <-> '(500,500)'::point)
mS
eCC
O on
N
fer
20
en
11
e c

32. Faster
Unlogged Tables
Background: every write to a table in Postgres is essentially *two*
writes because of the Write Ahead Log (basis for replication, and
durability guarantees)
Unlogged tables are FREED of this durability constraint!
Could think of them as Global Temporary tables (see Haas' post)
http://www.depesz.com/index.php/2011/01/03/waiting-for-9-1-unlogged-tables/
http://rhaas.blogspot.com/2010/05/global-temporary-and-unlogged-tables.html
Example:
=$ perl -le 'my $x = "aaaaaaaa"; for (1..450000) {print $x++}' |
perl -pe 's/.*/(047$&047)/;s/s*z/,/ if $.%3' |
sed 's/.*/insert into test (some_text) values &;/'
So
O mS
eCC
O on
N
fer
20
en
11
e c

33. Stronger Serializable
Snapshot Isolation
PostgreSQL's is the first production implementation
Includes: per transaction support
Why use it?
* No more blocking on table locks!
* No more "select for update"
* Simplifies programming
Wiki: http://wiki.postgresql.org/wiki/SSI
Developer:
My notes from Kevin's version of the talk: http://www.chesnok.com/
daily/2011/03/24/raw-notes-from-kevin-grittners-talk-on-ssi/
So
O mS
eCC
O on
N
fer
20
en
11
e c

34. Stronger
c e
11
en
20
fer
O on
N
eCC
O mS
So

35. Stronger
Stronger Foreign Data
Wrappers
* Remote datasource access
* Initially implemented TEXT, CSV data sources
* Twitter: http://pgxn.org/dist/twitter_fdw/
* Google Summer of Code student is working on other data
sources
* Underway currently: Oracle and MySQL sources
Awesome because:
* 6-function API for implementing
* Good for implementing imports, things which would otherwise
fail if you just
used COPY (imporoving COPY is a whole other rant...)
So
O mS
eCC
O on
N
fer
20
en
11
e c

36. Stronger
Stronger Foreign Data
Wrappers
Not awesome because:
* Nothing other than sequential scans are possible for query plans
* Not that many data sources implemented yet
Expect tons of FDWs to be implemented once we get 9.1 to
production release
So
O mS
eCC
O on
N
fer
20
en
11
e c

37. Stronger Syncronous
Replication
This is *the* most important use case for WAL-level replication.
User:
http://developer.postgresql.org/pgdocs/postgres/warm-
standby.html#SYNCHRONOUS-REPLICATION
(kind of terrible for a person trying to understand how it works)
Somewhat better:
http://wiki.postgresql.org/wiki/What
%27s_new_in_PostgreSQL_9.1#Synchronous_replication_and_other_replic
ation_features
Wiki: http://wiki.postgresql.org/wiki/Synchronous_replication
So
O mS
eCC
O on
N
fer
20
en
11
e c

38. Stronger Syncronous
Replication
WAL stands for "write ahead log" and it is what
ensures durability for all write operations
* WAL-shipping 16-MB chunks of these records
was the pre-9.0 method for built-in replication
* 9.0 and later have Streaming rep - per WAL
record (per-commit) shipping of changes
* Logical changes to DB, NOT statement-based
* All-or-nothing wrt databases and tables
So
O mS
eCC
O on
N
fer
20
en
11
e c

39. Stronger Syncronous
Replication
* Create a replication role (no longer a forced role
creation like in 9.0)
* pg_hba.conf (host based authorization) update to
allow replication user to access system
* Five key configuration options: wal_level,
max_wal_senders, archive_mode, archive_command,
synchronous_standby_names
* Create a base backup that will be the slave
* recovery.conf config: restore_command,
standby_mode, primary_conninfo, trigger_file
So
O mS
eCC
O on
N
fer
20
en
11
e c

40. Terrifyingly live demo.
So
O mS
eCC
O on
N
fer
20
en
11
e c

41. Harder Better
Faster Stronger
Thanks to
Josh Berkus
http://www.pgexperts.com/document.html?id=50
Depesz
http://www.depesz.com
So
O mS
eCC
O on
N
fer
20
en
11
e c

42. Harder Better
Faster Stronger
Other Postgres talks:
Pro PostgreSQL
Robert Treat, 1:40pm Wednesday
Unbreaking your Django Application
Christophe Pettus, 9am
So
O mS
(tutorial happening NOW)
eCC
O on
N
fer
20
en
11
e c

43. Harder Better
Faster Stronger
Selena Deckelmann
@selenamarie
http://chesnok.com
For hire! :)
So
O mS
eCC
O on
N
fer
20
en
11
e c