1.
Integrating PostgreSQL
Reuven M. Lerner • reuven@lerner.co.il
Rails Israel Conference 2013
October 9th, 2013 • Tel Aviv
1

2.
Who am I?
• Web developer since 1993
• Linux Journal columnist since 1996
• PostgreSQL user since at least 1999
• Ruby/Rails developer, teacher since 2005
• PhD candidate in learning sciences at
Northwestern University
2

3.
But before all of that...
I was a kid.
3

4.
(Yes, a geeky kid.)
But before all of that...
I was a kid.
3

5.
4

6.
5

7.
6

8.
7

9.
Databases?!?
Who needs them?
8

10.
Er, I did. And do.
• Whadaya know? Databases are useful!
• They offer an abstraction layer for our data
• I don’t care how it is stored, so long as the
data is safe, and I can get it back
9

11.
What is a database?
Database
Store data
confidently
Retrieve data
flexibly
10

12.
Relational databases
• Client-server
• All data is stored in two-dimensional tables
• Client-server communication is in SQL
• SQL is standard. Each database implements
a superset of a subset of that standard.
11

13.
“Impedance mismatch”
• You basically couldn’t find two languages
that are more different from one another
than Ruby and SQL:
• Ruby — object oriented, imperative
• SQL — table oriented, declarative
• Mixing them is ugly, as we all know
12

14.
ActiveRecord!
• ActiveRecord is the default ORM (object-
relational manager) in Rails
• ActiveRecord translates our Ruby method
calls to SQL queries
• Query results are turned into Ruby objects
and collections
13

15.
The good news...
• ActiveRecord papers over the differences
between databases!
14

16.
... and the bad news
• ActiveRecord papers over the differences
between databases!
15

17.
The really bad news
• This means we’re:
• Writing extra code
• Slowing down our apps
• Putting logic in the wrong place
• Database logic shouldn’t be in your app
(just like SQL shouldn’t be in your HTML)
16

18.
PostgreSQL
17

19.
PostgreSQL
17

20.
PostgreSQL
17

21.
PostgreSQL
17

22.
PostgreSQL
17

23.
PostgreSQL
17

24.
PostgreSQL
• Very fast, very scalable.
• Amazingly flexible, easily extensible.
• Rock-solid — no crashes, corruption,
major security issues for years
• Ridiculously easy administration
• It also happens to be free (MIT/BSD)
18

25.
Brief history
• Ingres (Stonebreaker, Berkeley)
• Postgres (Stonebreaker, Berkeley)
• PostgreSQL project = Postgres + SQL
• About one major release per year
• Latest and greatest is 9.3
19

26.
Features!
• MVCC
• Custom data types
• CTEs
• Functional, partial indexes
• Full-text search
• Server-side functions
20

27.
Rails 4 to the rescue!
• As of Rails 4, many PostgreSQL features are
supported, out of the box, by ActiveRecord
• The best of both worlds:Amazing database
features, and we don’t have to use SQL!
21

28.
But wait!
• What if I want to switch to another
database?
• Won’t I lose the platform independence?
• Yes, you will.
• Fortunately, this will almost certainly not
happen.
• And if it does, it’ll be super-painful anyway.
22

29.
Wait again!
• This means that I can’t use SQLite in
development, and PostgreSQL on the
server.
• Yes, that’s right.
• You should use the same database in
development and production. Please.
23

30.
Data types
• Databases are like programming languages:
The right choice of data type can have huge
consequences in speed, size, and flexibility
• PostgreSQL has some special ones that can
really come in handy!
24

31.
Network types
• PostgreSQL supports INET, CIDR, and
MACADDR types
• If you’re ever storing network addresses,
you should use these, not a text type!
• Savings in space (and time), and faster
comparisons (numeric vs. textual)
25

32.
CREATE TABLE Logins (
id SERIAL PRIMARY KEY,
local_ip INET NOT NULL,
remote_ip INET NOT NULL);
INSERT INTO Logins (local_ip, remote_ip) values
('192.168.1.1', '10.1.2.3');
reuven=# INSERT INTO Logins (local_ip, remote_ip) values
('999.999.999.1000', '10.1.2.3');
ERROR: invalid input syntax for type inet:
"999.999.999.1000"
LINE 1: INSERT INTO Logins (local_ip, remote_ip) values
('999.999.99...
26

33.
Rails 4 migrations
$ rails g model login local_ip:inet remote_ip:inet
class CreateLogins < ActiveRecord::Migration
def change
create_table :logins do |t|
t.inet :local_ip, nulls: :false
t.inet :remote_ip, nulls: :false
t.timestamps
end
end
end
27

34.
Rails 4 app
Login.create!(local_ip:'192.168.1.1', remote_ip:'10.1.2.3')
Login.create!(local_ip:'192.168.1.1000', remote_ip:'10.1.2.3')
IPAddr::InvalidAddressError: invalid address
i = Login.first.local_ip
i.class
=> IPAddr
i.ipv4?
=> true
i.ipv6?
=> false
28

35.
Managing a network?
• We also have CIDR and MAC addresses!
Netconfig.create!(net: '192.168.1.1/255.255.255.0')
29

36.
UUIDs
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE TABLE evil_companies (
id UUID NOT NULL PRIMARY KEY DEFAULT uuid_generate_v1(),
name TEXT);
INSERT INTO evil_companies (name) VALUES ('HOT Internet');
INSERT INTO evil_companies (name) VALUES ('HOT TV');
INSERT INTO evil_companies (name) VALUES ('HOT Telephone');
INSERT INTO evil_companies (name) VALUES ('Voldemort, Ltd.');
30

37.
Wait... extensions?
• Yes, PostgreSQL has extensions!
• Download and install them from PGXN
• (Think of them as gems for PostgreSQL)
• Some, like uuid-ossp, come by default
• Activate them with CREATE EXTENSION
31

38.
32

39.
With this in place...
SELECT * FROM evil_companies ;
┌──────────────────────────────────────┬─────────────────┐
│ id │ name │
├──────────────────────────────────────┼─────────────────┤
│ da7a333a-3069-11e3-b4fd-28cfe91f81e7 │ HOT Internet │
│ dc480be2-3069-11e3-a0f4-28cfe91f81e7 │ HOT TV │
│ ddadabfe-3069-11e3-af1d-28cfe91f81e7 │ HOT Telephone │
│ 4c33a128-306a-11e3-8000-28cfe91f81e7 │ Voldemort, Ltd. │
└──────────────────────────────────────┴─────────────────┘
33

40.
Migrations
# Requires some manual tinkering
class CreateEvilCompanies < ActiveRecord::Migration
def change
enable_extension "uuid-ossp"
create_table :evil_companies, id: :uuid do |t|
t.text :name
t.timestamps
end
end
end
34

41.
Migrations
# Requires some manual tinkering
class CreateEvilCompanies < ActiveRecord::Migration
def change
enable_extension "uuid-ossp"
create_table :evil_companies, id: :uuid do |t|
t.text :name
t.timestamps
end
end
end
34

42.
In the app
EvilCompany.create!(name: 'HOT Internet')
EvilCompany.first.id
=> "07d3f537-c008-4006-9f2b-0592c7df3ebf"
35

43.
JSON
• Remember the “impedance mismatch”
between objects and databases?
• Now we have another one — we get data
in JSON, turn it into database objects, and
then turn it back into JSON
• But now PostgreSQL can store, retrieve,
and query JSON!
36

44.
CREATE TABLE bad_service (
id SERIAL,
info JSON
);
INSERT INTO bad_service (info)
values ('{"company":"HOT Internet", "badness":9}');
INSERT INTO bad_service (info) VALUES ('la la la');
ERROR: invalid input syntax for type json
LINE 1: insert into bad_service (info) values ('la la la');
^
DETAIL: Token "la" is invalid.
CONTEXT: JSON data, line 1: la...
37

45.
JSON operators!
• As of PostgreSQL 9.3, we get operators
• Work with JSON, much like XML:
SELECT info#>'{company}' from bad_service;
┌────────────────┐
│ ?column? │
├────────────────┤
│ "HOT Internet" │
└────────────────┘
38

46.
Migration
$ rails g model bad_service info:json
class CreateBadServices < ActiveRecord::Migration
def change
create_table :bad_services do |t|
t.json :info
t.timestamps
end
end
end
39

47.
App
BadService.create!(info: '{"company":"HOT Internet"}')
b = BadService.first
b.info.class
=> Hash
b.info['company']
=> "HOT Internet"
40

48.
Ranges
CREATE TABLE forecast (
forecast_for DATE,
hi_lo int4range);
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('8-
oct-2013', '[19,29]');
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('9-
oct-2013', '[20,32]');
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('10-
oct-2013', '[19,28]');
41

49.
SELECT * FROM forecast ;
┌──────────────┬─────────┐
│ forecast_for │ hi_lo │
├──────────────┼─────────┤
│ 2013-10-08 │ [19,30) │
│ 2013-10-09 │ [20,33) │
│ 2013-10-10 │ [19,29) │
└──────────────┴─────────┘
SELECT * FROM forecast WHERE hi_lo @> 19;
┌──────────────┬─────────┐
│ forecast_for │ hi_lo │
├──────────────┼─────────┤
│ 2013-10-08 │ [19,30) │
│ 2013-10-10 │ [19,29) │
└──────────────┴─────────┘
42

50.
Migration
$ model forecast forecast_for:date hi_lo:int4range
class CreateForecasts < ActiveRecord::Migration
def change
create_table :forecasts do |t|
t.date :forecast_for
t.int4range :hi_lo
t.timestamps
end
end
end
43

51.
Ranges in apps
Forecast.create!(forecast_for: Time.now, hi_lo: 20..30)
Forecast.first.hi_lo
=> 20...31
Forecast.first.hi_lo.class
Range
44

52.
Arrays!
CREATE TABLE posts (
body text,
tags text[]
);
INSERT INTO posts (body, tags) VALUES ('Hello, world',
'{intro, hello}');
INSERT INTO posts (body, tags) VALUES ('Hello again',
'{DRY, boring, hello}');
45

53.
SELECT * FROM posts;
┌──────────────┬────────────────────┐
│ body │ tags │
├──────────────┼────────────────────┤
│ Hello, world │ {intro,hello} │
│ Hello again │ {DRY,boring,hello} │
└──────────────┴────────────────────┘
SELECT * from posts where 'intro' = ANY(tags);
┌──────────────┬───────────────┐
│ body │ tags │
├──────────────┼───────────────┤
│ Hello, world │ {intro,hello} │
└──────────────┴───────────────┘
46

54.
Arrays in migrations
$ rails g model post body:text tags:text
class CreatePosts < ActiveRecord::Migration
def change
create_table :posts do |t|
t.text :body
t.text :tags, array: true
t.timestamps
end
end
end
47

55.
Arrays in your app
Post.create!(body: 'First post!', tags: %w(first second
third))
Post.first.tags
=> ["first", "second", "third"]
48

56.
Normalization = DRY
• Everyone loves to talk about tagging as a
great example of PostgreSQL arrays
• Um... don’t forget about normalization, the
DRY of database design
• I know, it’s not cool to talk about it in the
Age of NoSQL. But it really does work.
49

57.
Premature
denormalization is the
root of all database evil.
50

58.
Hstore
• A key-value storage system in PostgreSQL!
• Any column can be defined as hstore
• Then you can query it — and you get a
hash back!
• It’s sort of like Redis or memcached, but
integrated into (and backed by) Postgres!
51

59.
PostgreSQL
CREATE EXTENSION hstore;
CREATE TABLE posts2 (
body text,
tags hstore
);
INSERT INTO posts2 (body, tags)
VALUES ('Hello, there', '"hello" => 2, "boring" => 10');
INSERT INTO posts2 (body, tags)
values ('Hello again', '"DRY" => 5,
"hello" => 2, "boring" => 3');
52

60.
SELECT * from posts2 where defined(tags, 'hello');
┌──────────────┬─────────────────────────────────────────┐
│ body │ tags │
├──────────────┼─────────────────────────────────────────┤
│ Hello, there │ "hello"=>"2", "boring"=>"10" │
│ Hello again │ "DRY"=>"5", "hello"=>"2", "boring"=>"3" │
└──────────────┴─────────────────────────────────────────┘
(2 rows)
SELECT tags -> 'boring' from posts2 ;
┌──────────┐
│ ?column? │
├──────────┤
│ 10 │
│ 3 │
└──────────┘
(2 rows)
53

61.
And also...
SELECT * FROM posts2
WHERE (tags -> 'boring')::integer > 5;
┌──────────────┬──────────────────────────────┐
│ body │ tags │
├──────────────┼──────────────────────────────┤
│ Hello, there │ "hello"=>"2", "boring"=>"10" │
└──────────────┴──────────────────────────────┘
54

62.
Migration
$ rails g model newpost body:text tags:hstore
class CreateNewposts < ActiveRecord::Migration
def change
enable_extension "hstore"
create_table :newposts do |t|
t.text :body
t.hstore :tags
t.timestamps
end
end
end
55

63.
In your app
Newpost.create!(:body => 'foo', :tags => {a:1, b:2})
Newpost.first.tags
=> {"a"=>"1", "b"=>"2"} # hash!
Newpost.first.tags['a']
=> "1" # now a string!
Newpost.first.tags[:a]
=> nil # not WithIndifferentAccess!
56

64.
Indexes
• PostgreSQL offers different index types
• Specify these in your migrations
• Partial indexes (with a WHERE clause)
• Functional indexes (apply a function!)
• Full-text indexes (many languages)
• Geospatial indexes (install PostGIS!)
57

65.
Some things
are still missing
• “The Active Record way claims that
intelligence belongs in your models, not in
the database.As such, features such as
triggers or foreign key constraints, which
push some of that intelligence back into the
database, are not heavily used.”
• — Rails Guide,ActiveRecord Migrations
58

66.
Sigh.
59

67.
60

68.
Your data are your
crown jewels!
61

69.
Summary
• Rails good! PostgreSQL good!
• Rails + PostgreSQL = Doubleplus good!
• Rails 4 + PostgreSQL = Secure storage,
flexible retrieval, and a lot of geeky DB fun!
62

70.
Thanks!
(Any questions?)
reuven@lerner.co.il
http://www.lerner.co.il/
@reuvenmlerner
054-496-8405
“reuvenlerner” on Skype
63