13. Memory
flush
index
cleanup k1 c1:v c2:v c3:v
k2 c1:v c2:v
SSTable
Hard drive
14. “Java is a memory hog”
✤ Large overhead for typical objects and collections
✤ How large?
✤ java.lang.instrument.Instrumentation
✤ JAMM: Java Agent for Memory Measurements
✤ https://github.com/jbellis/jamm
15. org.apache.cassandra.cache.SerializingCache
✤ Live objects are about 85% JVM bookeeping
✤ org.apache.cassandra.cache.FreeableMemory using reference
counting
✤ Considering doing reference-counted, off-heap memtables
as well
18. m[un]map
✤ Log-structured storage wants to remove old files post-
compaction; some platforms disallow deleting open files
✤ Old workaround (pre-1.0):
✤ use PhantomReference to tell when mmap’d file is GC (hence
unmapped)
✤ Poor user experience and messy corner cases
✤ New workaround:
✤ Class.forName("sun.nio.ch.DirectBuffer").getMethod("cleaner")
19. mmap part 2
✤ 2GB limit via ByteBuffer:
public abstract byte get(int index)
✤ Workaround: MmappedSegmentedFile
public Iterator<DataInput> iterator(long position)
20. link
✤ Used for snapshots
✤ Old workaround: JNA
✤ New workaround: supported directly by Java7
21. mlockall
✤ swappiness: pissing off database developers since 2001 (?)
✤ mlockall(MCL_CURRENT)
23. A plug for JNA
✤ https://github.com/twall/jna
static {
try {
Native.register("c");
...
private static native int mlockall(int flags)
throws LastErrorException;
24. The fallacy of choosing portability over power
✤ Applets have been dead for years
✤ Python gets it right
✤ import readline
25. The fallacy of choosing safety over power
✤ Allowing munmap would expose developers to segfaults
✤ But, relying on the GC to clean up external resources is a
well-known antipattern
✤ File.close
✤ We need munmap badly enough that we resort to
unnatural and unportable code to get it
✤ You haven’t kept us from risking segfaults, you’ve just made us
miserable
29. Still true
✤ "Many concurrent algorithms are very easy to write with a
GC and totally hard (to down right impossible) using
explicit free." -- Cliff Click