3. Cassandra 0.6 & 0.7
Jonathan Ellis
@spyced
Tuesday, August 10, 2010
4. Quiet change of policy
• 0.5.1 was bug fixes only
• Too early to be strict about bugfix-only
policy in stable branch, especially w/ 0.7
being longer/more break-y
• Maybe after 1.0?
Tuesday, August 10, 2010
5. 1500
mails sent
1125
750
375
0
Jan Feb Apr May Jun Jul
(0.5) (0.5.1) Mar (0.6, 0.6.1) (0.6.2) (0.6.3) (0.6.4)
Tuesday, August 10, 2010
6. Lots of bug fixes
• 85 issues marked Resolved/Fixed in 0.6
branch after 0.6 released
Tuesday, August 10, 2010
7. Runtime configuration
• concurrent reads, writes (0.6.2)
• making it easier to bandage your foot after you
shoot it
• PhiConvictThreshold (0.6.2)
Tuesday, August 10, 2010
8. Performance
• JVM GC defaults (0.6.2)
• Faster commitlog (0.6.2)
• Faster range slice, Hadoop jobs (0.6.1, 2)
• Better parallelization of multiget (0.6.4)
• UTF8Type, UUIDType optimizations (0.6.5)
Tuesday, August 10, 2010
10. Hinted Handoff
• 0.6.0: send hints to natural replicas
• 0.6.0: fix row-level concurrency bottleneck
• 0.6.2: option to disable entirely
• 0.6.3: remove hourly scan
• 0.6.4: lower priority
• 0.6.5: paging of large hinted rows
• 0.7.0: large rows
Tuesday, August 10, 2010
11. Why keep HH around?
https://www.cloudkick.com/blog/2010/jan/12/visual-ec2-latency/
Tuesday, August 10, 2010
12. Compaction priority
-XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42
-Dcassandra.compaction.priority=1
Extended to HH in 0.6.4
Tuesday, August 10, 2010
14. JMX for bloom filters
• o.a.c.db:ColumnFamilyStores
• getBloomFilterFalsePositives
• [not in nodetool yet]
Tuesday, August 10, 2010
15. Flow control in 0.5
• Why backpressure doesn’t fit Cassandra
Tuesday, August 10, 2010
16. Flow Control in 0.6.4
• Replica nodes drop hopeless requests on
the floor
• Coordinator node is unaffected
• TimedOutException signals client to back off
• Requires enough memory to buffer
RPCTimeout’s worth of requests
• (In the short term, you’re still screwed)
Tuesday, August 10, 2010
17. Flow Control, 0.6.4
IncomingTcpConnection
Message Deserializer Uncapped
Read Mutation Capped at 4096
Tuesday, August 10, 2010
20. Dynamic snitch
• sortByProximity
Tuesday, August 10, 2010
21. Open problems
• Linux/mmap/swap unholy trio (0.6.5)
• Memory fragmentation (0.6.5?)
• Compaction effect on caches (0.7.1?)
Tuesday, August 10, 2010
22. mmap and swap
• The problem
• Mitigations
• mmap_index_only
• swappiness=0
• turn off swap
• mlockall at startup (Xms=Xmx)
Tuesday, August 10, 2010
23. GC Fragmentation
• Culprit of infamous CASSANDRA-1014?
• Mitigation: tune with much larger new
generation / tenuring threshold?
Tuesday, August 10, 2010
24. Compaction and caches
• Compactions wrecks the OS fs cache
• Wrecks Cassandra key cache, too
• (but not row cache)
Tuesday, August 10, 2010
31. W A
F
(A-F]
(A-F]
T
(F-L]
L
Tuesday, August 10, 2010
32. W A
F
Data
T
L
Index
Filter
Tuesday, August 10, 2010
33. Streaming in 0.7
W A
F
T
L
Index
Filter
Tuesday, August 10, 2010
34. DatacenterStrategy
• RackAwareStrategy is tuned for 3 replicas
and 2 data centers
• DS allows configuring replicas per data
center, per Keyspace
Tuesday, August 10, 2010
35. Minor features in 0.7
• read_repair_chance
• per-keyspace request scheduling
• Hadoop OutputFormat
• Per CF what used to be global
(gc_grace_seconds, memtable thresholds)
Tuesday, August 10, 2010
36. 0.7 API changes
• String keys become byte[]
• Thrift keyspace argument moved to
set_keyspace
• i64 timestamp becomes Clock
• SlicePredicate for _count methods
Tuesday, August 10, 2010
37. 0.7 performance
• Reads roughly 100% faster, thanks largely to
removing String creation
• Row-cached reads up to 8x faster after
optimizations by tjake and jbellis
• Optimizations for reads of large rows
• 0.7.1? ~20% improvement everywhere from
Thrift optimizations
Tuesday, August 10, 2010
38. Thrift
• OOMs on malformed packets
• Python Unicode string issues
• PHP support is buggy and maintainerless
Tuesday, August 10, 2010
39. After 0.7.0
• IndexOperator.GT
• Triggers / plugins
• Avro?
• On-disk data format improvements
(Compression, heirarchical data?)
• Auth
Tuesday, August 10, 2010
