This is a workshop that Mike Wynholds and Rudy Jahchan from Carbon Five, with special guest Ryan King from Twitter, gave at SXSW 2011.

1. Embracing NoSQL
Your First Cassandra Project
#embracingnosql
@mwynholds
@rudy
@rk

2. Goal

3. Goal
By the time you leave
here, you will be
comfortable using
Cassandra in Rails

4. GENERALIZATION WARNING:
PRESENTATION CAN CAUSE
BROAD SIMPLIFICATIONS,
OPINIONS, NIT-PICKING, AND MAY
COMPLICATE PROGRESS.

5. Schedule
• 3:30 - 5:00 Hands On Workshop
• Introduction
• Setup
• Reading and Writing
• Timeline
• ORM
• Denormalization
• After 5ish - Panel Q&A

6. Why NoSQL?
Scalability for the
social web

7. CAP

8. CAP
Consistency

9. CAP
Consistency
Availability

10. CAP
Consistency
Partition
Availability
Tolerance

11. CAP
Consistency
Partition
Availability
Tolerance
You can only have two at a time!

12. CAP
C
P A

13. CAP
C
MongoDB
HBase
Redis
P A

14. CAP
C
MongoDB
MySQL
HBase
Postgres
Redis
P A

15. CAP
C
MongoDB
MySQL
HBase
Postgres
Redis
P A
CouchDB
Riak

16. CAP
C
MongoDB
MySQL
HBase
Postgres
Redis
P A
CouchDB
Riak
Cassandra!

17. Cassandra

18. Cassandra
http://cassandra.apache.org

19. Column Oriented
123
date:
1/23/11
title:
My Title
url:
http://...
user:
rudy
...
546
body:
Lipsum...
date:
3/13/11
status:
pending
title:
Foo
user:
jared
...
680 date:
1/23/11
url:
http://...
user:
rob
...
791 body:
Lipsum...
date:
3/13/11
title:
Bar
user:
mike
...

20. Decentralized &
Distributed

21. Decentralized &
Distributed
Node

22. Decentralized &
Distributed
Node
Node Node
Node

23. Decentralized &
Distributed
Sync Node
Sync
Node Node
Node
Sync Sync

24. Decentralized &
Distributed
Sync Node
Sync
Write
Node Node
Node
Sync Sync

25. Decentralized &
Distributed
Sync Node
Sync
Write
Node Node
Read
Node
Sync Sync

26. Tunably Consistent
Write
ONE
Read

27. Tunably Consistent
where n = replication factor
Write
QUORUM
(2n +1)
Read

28. Tunably Consistent
Write
ALL
Read

29. So How Much Faster Is
Cassandra?

30. So How Much Faster Is
Cassandra?


31. So How Much Faster Is
Cassandra?


32. No Really, how fast?

33. No Really, how fast?
w/ 50GB MySql Cassandra
Write 300ms 0.12ms
Read 350ms 15ms

34. First NoSQL Rule

35. First NoSQL Rule
Don’t Use NoSQL
Right Away!

36. Build Your
Application First,
THEN Use NoSQL
As Needed!

37. What We’re Building
Today

38. What We’re Building
Today
Reddit

39. What We’re Building
Today
Reddit
+ Digg

40. What We’re Building
Today
Reddit
+ Digg
- Quite a Lot

41. What We’re Building
Today
Reddit
+ Digg
- Quite a Lot
DUGGIT!

42. Login

43. Submit Links

44. Vote on Links

45. LAB: Installation
• Install Cassandra
• Start it up
• Command line interface
• Shut it down

46. Lexicon
Database Keyspace
Tables
Column Families

47. Rows and Columns
date: title: url:
123
1/23/11 My Title http://...
body: date: status: title:
546
Lipsum... 3/13/11 pending Foo
680 date: url:
1/23/11 http://...
body: date: title:
791
Lipsum... 3/13/11 Bar

48. Rows and Columns
date: title: url:
123
1/23/11 My Title http://... Row
body: date: status: title:
546
Lipsum... 3/13/11 pending Foo
680 date: url:
1/23/11 http://...
body: date: title:
791
Lipsum... 3/13/11 Bar

49. Rows and Columns
date: title: url:
123
1/23/11 My Title http://... Row
body: date: status: title:
546
Lipsum... 3/13/11 pending Foo
Row 680 date: url:
Key 1/23/11 http://...
body: date: title:
791
Lipsum... 3/13/11 Bar

50. Rows and Columns
date: title: url:
123
1/23/11 My Title http://... Row
body: date: status: title:
546
Lipsum... 3/13/11 pending Foo
Row 680 date: url:
Column
Key 1/23/11 http://...
body: date: title:
791
Lipsum... 3/13/11 Bar

51. In SQL Columns Exist
for Every Row ...
id title body url date user
123 My Title http://... 1/23/11 rudy
456 Foo Lipsum.. 3/13/11 jared
680 http://... 1/23/11 rob
791 Bar Lipsum.. 3/13/11 mike

52. ... But NOT in Column
Families!
date: title: url: user:
123
1/23/11 My Title http://... rudy
body: date: title: user:
546
Lipsum... 3/13/11 Foo jared
680 date: url: user:
1/23/11 http://... rob
body: date: title: user:
791
Lipsum... 3/13/11 Bar mike

53. In SQL New Columns
Require ALTER TABLE...
id title body url date user status
123 My Title http://... 1/23/11 rudy
456 Foo Lipsum.. 3/13/11 jared pending
680 http://... 1/23/11 rob
791 Bar Lipsum.. 3/13/11 mike

54. ... But Are Simple
Writes in Cassandra!
date: title: url: user:
123
1/23/11 My Title http://... rudy
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
680 date: url: user:
1/23/11 http://... rob
body: date: title: user:
791
Lipsum... 3/13/11 Bar mike

55. What’s a Column?
user: Column Name
rudy Value

56. What’s a Column?
user: Column Name
rudy Value
1234 Timestamp

57. Columns are Ordered
body: date: title: user:
546
Lipsum... 3/13/11 Foo jared

58. Columns are Ordered
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared

59. Name can be Anything!
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
11/11/03: 11/12/03: 11/13/03: 11/14/03:
rudy Party Recover Panel Return
123: 456: 680: 791:
c4af67
7 0 2 5

60. PATTERN: Simple Index
f(x, y) = z
y1: y2: y3:
x1
z11 z12 z13
x2 y2: y4:
z22 z24
y1: y3: y4:
x3
z31 z33 z34

61. Duggit:Votes
f(link_id, user_id) = vote
jared: rob: rudy:
123
1 -1 -1
680 mike: rob:
1 1
jared: mike: rudy:
791
-1 1 1

62. LAB: Setup
• Create duggit keyspaces
• Create votes column family
• Create some sample data

63. Cassandra API
Node

64. Cassandra API
Thrift
Node

65. Cassandra API
JAVA Ruby Python WhatEvs
Thrift
Node

66. Use High Level Client
cassandra
Ruby
Thrift
Node
github.com/fauna/cassandra

67. Connecting
con = Cassandra.new(keyspace, address)

68. Connecting
con = Cassandra.new('duggit', '127.0.0.1:9160')

69. Reading a Row
row = con.get(column_family, key)

70. Reading a Row
link_id = ‘123’
row = con.get(:votes, link_id)
url = row['url']

71. Reading Many Rows
results = con.multi_get(column_family, keys)

72. Reading Many Rows
link_ids = [‘123’, ‘456’]
results = con.multi_get :votes, keys
results.each do |key, row|
row_key = key
url = row['url']
end

73. Writing
con.insert(column_family, key, column_values)

74. Writing
link_id = ‘123’
vote = {user.id => vote_val}
con.insert :votes, link_id, vote

75. LAB: Simple Column
Family
• Connect to Cassandra
• Rewrite Vote
• Rewrite Link#value
• Configure testing frameworks

76. Querying?
Not Really ...

77. Slice
con.get_columns(column_family,
key,
:columns => column_names
con.get_columns(:links,
'546',
:columns => ['date', 'user']
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared

78. Range
con.get(column_family,
row_key,
:start => start,
:finish => finish)
con.get(:link, '546', :start => 'd', :finish => 'u')
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared

79. REMEMBER: Column
Names can Be Anything
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
11/11/03: 11/12/03: 11/13/03: 11/14/03:
rudy Party Recover Panel Return
123: 456: 680: 791:
c4af67
7 0 2 5

80. PATTERN: Timeline
Columns are timestamps
rudy
mike
jared

81. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330:
rudy
OHAI
mike
jared

82. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330:
rudy
OHAI
mike: Hi
mike 0335:
Hi
jared

83. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330:
rudy
OHAI
mike: Hi
jared:YO 0335:
mike
Hi
0345:
jared
YO

84. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346:
rudy
OHAI HALP
mike: Hi
jared:YO 0335:
mike
rudy:HALP Hi
0345:
jared
YO

85. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346: 0415:
rudy
OHAI HALP PLZ
mike: Hi
jared:YO 0335:
mike
rudy:HALP Hi
rudy: PLZ
0345:
jared
YO

86. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346: 0415:
rudy
OHAI HALP PLZ
mike: Hi
jared:YO 0335:
mike
rudy:HALP Hi
rudy: PLZ
0345: 0420:
jared
jared: RTFM YO RTFM

87. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346: 0415:
rudy
OHAI HALP PLZ
mike: Hi
jared:YO 0335:
mike
rudy:HALP Hi
rudy: PLZ
0345: 0420: 0445:
jared
jared: RTFM YO RTFM NEWB
jared: NEWB

88. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346: 0415:
rudy
OHAI HALP PLZ
mike: Hi
jared:YO 0335: 0515:
mike
rudy:HALP Hi LOL
rudy: PLZ
0345: 0420: 0445:
jared
jared: RTFM YO RTFM NEWB
jared: NEWB
mike: LOL

89. PATTERN: Timeline
Columns are timestamps
rudy: OHAI 0330: 0346: 0415: 0645:
rudy KTHXBAI
OHAI HALP PLZ
mike: Hi
jared:YO 0335: 0515:
mike
rudy:HALP Hi LOL
rudy: PLZ
0345: 0420: 0445:
jared
jared: RTFM YO RTFM NEWB
jared: NEWB
mike: LOL
rudy: KTHXBAI

90. Duggit: Users Links
20110101: 20110214: 20110311: 20110313:
rudy 123 311 777 987
20101211: 20110113: 20110225: 20110301 20110314:
jared
4 202 456 531 1066
20101231: 20110228: 20110304:
rob
99 432 680
20101211: 20101225: 20110307: 20110312:
mike 791
500 53 696

91. “Pagination”
con.get ‘jared’, :count => 3, :reverse => true
20101211: 20110113: 20110225: 20110301 20110314:
jared 531
4 202 456 1066
con.get ‘jared’, :count => 3,
:reverse => true, :start => 20110224
20101211: 20110113: 20110225: 20110301 20110314:
jared 531
4 202 456 1066

92. But What about Across
Rows?
20110101: 20110214: 20110311: 20110313:
rudy 311
123 777 987
20101211: 2011013: 20110225: 20110301 20110314:
jared 531
4 202 456 1066
shuffle
20110214: 20110225: 20110301 20110311: 20110313: 20110314:
311 456 531 777 987 1066

93. “All” Timeline
All
20101211:
4
20101231:
99
20110101:
123
20110113:
202
20110214:
311
...
20110101: 20110214: 20110311: 20110313:
rudy 311
123 777 987
20101211: 20110113: 20110225: 20110301 20110314:
jared 531
4 202 456 1066
20101231: 20110228: 20110304:
rob 99 432 680

94. The Balance
rudy
rob
All mike
jared

95. The Balance
rudy
rob
All mike
jared

96. Batch Writing
con.batch do
con.insert :user_links, user.id, value
con.insert :user_links, 'All', value
end

97. LAB: Timelines
• Create a user_links column family
• Write links to appropriate timelines
• Return the recent items

98. Still Hitting SQL
App Cassandra SQL

99. Pattern: ORM
Row = Object
Column = Object property
date: title: url: user:
123
1/23/11 My Title http://... rudy
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
680 date: url: user:
1/23/11 http://... rob

100. Secondary Index
Create an index on date column
date: title: url: user:
123
1/23/11 My Title http://... rudy
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
680 date: url: user:
1/23/11 http://... rob

101. Secondary Index
Query: where date = 1/23/11
date: title: url: user:
123
1/23/11 My Title http://... rudy
body: date: status: title: user:
546
Lipsum... 3/13/11 pending Foo jared
680 date: url: user:
1/23/11 http://... rob

102. LAB: ORM
• Rewrite Link to save to Cassandra
• Update supporting models and controllers

103. Still Two Reads
App Cassandra Cassandra

104. What if it isn’t there?

105. What if it isn’t there?
Write

106. What if it isn’t there?
Write

107. What if it isn’t there?
Link.recent_ids
Write

108. What if it isn’t there?
Link.recent_ids
Write
Link.by_ids

109. PATTERN:
Denormalization
Just like SQL:
Store it multiple times,
multiple locations

110. Seen it before ...
All
20101211:
4
20101231:
99
20110101:
123
20110113:
202
20110214:
311
...
20110101: 20110214: 20110311: 20110313:
rudy 311
123 777 987
20101211: 20110113: 20110225: 20110301 20110314:
jared 531
4 202 456 1066
20101231: 20110228: 20110304:
rob 99 432 680

111. But Different Column
Family Structures?
links
date: title: url: user:
123
1/23/11 My Title http://... rudy
users_to_links
20110101: 20110214: 20110311: 20110313:
rudy 311
123 777 987

112. Super-Column Families
All 20101211: 20101231: 20110101: 20110113: 20110214:
...
rudy 20110313: 20110311: 20110214: 20110101:
jared 20110314: 20110301: 20110225: 20110113: 20101211:
rob
20110304: 20110228: 20101231:

113. Super-Column
date: title: url: user:
20110314: = 20110314:
1/23/11 My Title http://... rudy

114. Write Object to Each
uuid = SimpleUUID::UUID.new
@id = uuid.to_guid
value = { 'id' => @id, 'title' => @title, ... }
pointer = { uuid => value }
con.batch do
con.insert :links, @id, value
con.insert :user_links, @user_id.to_s, pointer
con.insert :user_links, 'all', pointer
end

115. LAB: Denormalize Links
• Rewrite Link#save

116. Resources
• Wiki: http://wiki.apache.org/cassandra/
• Mailing List: user@cassandra.apache.org
• “Cassandra: The Definitive Guide” by Eban
Hewitt (@ebenhewitt)
• Presentations of Mike Malone
(@mjmalone)

117. Advanced Topics
• Deployment
• Cassandra in the Cloud
• Row Partitioning
• Compound Keys
• Other Strategies