Slides used for my presentation to the Austin Cassandra Meetup where I discuss how Cassandra fits in to Rackspace Cloud Monitoring. Hint: It's just a small part.

1. How we use
Cassandra
Gary Dusbabek
@gdusbabek

2. We’re hiring
http://jobs.rackspace.com/

3. http://www.flickr.com/photos/ollesvensson/4252196844/
Goals
CM Overview
Control Cluster
Data Cluster

4. CM Overview
Thousands of servers
Pre-existing solutions
Lessons learned from
Cloudkick
Internal versus external
Millions of checks
http://www.flickr.com/photos/jean_koulev/2697677595/

5. Cassandra
is the
least
interesting
part

6. Terminology
Entity
Something with an IP address or
host name

7. Terminology
Check
Tied to an Entity
Is an action
Produces metrics

8. Features
Remote checks
Collectors in 5 DCs
Processing in 3 DCs
Alerting
Notifications

9. All REST all
the time
More Ops friendly
Features Metrics
Agent
https://github.com/racker/virgo

10. Future
Automation
Prediction
Support hooks
Agent expansion
Correlation
Aggregation
Entity Spanning

11. 1,000 Words

12. Control
Cluster
Metadata
State
Three datacenters
High RF
Wide rows
Easy dump & load
https://github.com/racker/cassandra-syncer

13. Data Model
Rich but simple
Objects used together stored together
Simple parent-child relations
One row per customer (tenant)
Composite column names

14. Data Model
Good:
Single Parent/Child
Acyclic

15. Data Model
Bad:
Complex
Cyclic

16. Data Model
As Columns
Easy slicing

18. Y U NO MySQL?

19. Control Cluster
API server is Node.js
Javascript ORM library
• Define object model in JS
• Read/write entire objects
• Never think about CQL
node-cassandra-client
https://github.com/racker/node-cassandra-client

20. Control Cluster

21. The fun
starts Data
here Cluster

22. Data Cluster
Goal: Fast graphs
Time series data
Fewer data points
OK to shave resolution
Recent data is most important

23. Locator
Identifies a single metric
check identifier + name
E.g. my:check:id:ttfb

24. Granularity
Full, 5m, 20m, 60m, 240m,
1440m

25. Rollup Concepts
Slot (Range)
• Pegged at 4032 slots
• One slot is a range of seconds (varies
with granularity)
• metrics_locator CF
• Key is granularity name + slot num
• Columns index keys in rollup tables

26. Keyed by ascii
Bigint column names
Blob column values
JDBC
Rollups

27. Full Resolution! Arrival
• time, name, several metrics
• metric = name, type, value
• Compute locator and slot
• Insert metrics col=timestamp,
value=encoded metric
• Single Cassandra APPLY BATCH;

28. Rollups
• Two types
– Rollup all metrics from timeX to timeY
– Rollup a single metric from timeX to timeY
– Times may span multiple slots (ranges)
• Use rollups to produce rollups
– E.g.: use 20m data points to create 60m point.
– Store number of data points with rollup

29. Rollups
• Gotchas!
– Do not want to rollup a coarse range when finer
range that feeds data to it is scheduled for rollup
shortly
60m | | | …
20m | | | | | | | |…
.
5m |||||||||||||||||||||||||||||…
– Mind the “tail” during datapoint queries (calculate
rollups on the fly)

30. It Scales
Rollup operations are idempotent*
Simplifies availability
Rollups are easily parallelized
Hash partition the locator space

31. But…
What if data arrives after rollup
is performed?
More than 24hrs late: don’t
care, forget it
Else treat normally: slots are
scheduled for rollups as they
age

32. http://www.flickr.com/photos/nateone/3768979925
/
HBase
@gdusbabek

33. Image Credits
soccer http://www.flickr.com/photos/ollesvensson/4252196844/
apples http://www.flickr.com/photos/jean_koulev/2697677595/
bored http://www.flickr.com/photos/aidanmorgan/3709199304/
clock http://www.flickr.com/photos/szift/3196084839/
hallway http://www.flickr.com/photos/orinrobertjohn/4354716077/
car http://www.flickr.com/photos/ellenm1/3541270451
control http://www.flickr.com/photos/germanium/117612088/
model http://www.flickr.com/photos/mikeschinkel/2703438152/
robot http://www.flickr.com/photos/kb35/430976324/
swings http://www.flickr.com/photos/pierrebedat/1095337445/
cluster http://www.flickr.com/photos/inl/5097547405/
dish http://www.flickr.com/photos/mrpbps/2862208028/
sand http://www.flickr.com/photos/backkratze/3480338854/
slots http://www.flickr.com/photos/andresrueda/2925383781
sushi http://www.flickr.com/photos/basykes/4348613931/
airplane http://www.flickr.com/photos/fhashemi/72489620/
scale http://www.flickr.com/photos/puuikibeach/4765115333
hourglass http://www.flickr.com/photos/aidanmorgan/2331754875/
street http://www.flickr.com/photos/nateone/3768979925/