Introduces bookmarking collection, a GC algorithm that works with the virtual memory manager to eliminate paging. Just before memory is paged out, the collector "bookmarks" the targets of pointers from the pages. Using these bookmarks, BC can perform full garbage collections without loading the pages back from disk. By performing in-memory garbage collections, BC can speed up Java programs by orders of magnitude (up to 41X).

1. Garbage Collection
Without Paging
Matthew Hertz, Yi Feng,
Emery Berger
University of Massachusetts Amherst
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

2. Garbage Collection Performance
Garbage collection now performs
reasonably well
High throughput
Low pause times
Given large heap and sufficient memory
But: what happens when there’s not
enough RAM?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

3. GC Performance While Paging
RAM
Hard Disk
Heap: most pages in RAM, one on disk
GC begins
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

4. GC Performance While Paging
RAM
Hard Disk
Collector touches an evicted page...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

5. GC Performance While Paging
RAM
Hard Disk
... bringing the page into memory ...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

6. GC Performance While Paging
RAM
Hard Disk
... but triggers another page eviction.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

7. GC Performance While Paging
RAM
Hard Disk
... but triggers another page eviction.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

8. GC Performance While Paging
RAM
Hard Disk
Collector touches newly-evicted page...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

9. GC Performance While Paging
RAM
Hard Disk
Collector touches newly-evicted page...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

10. GC Performance While Paging
RAM
Hard Disk
... leading to the eviction of yet another page…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

11. GC Performance While Paging
RAM
Hard Disk
… which eventually triggers more paging.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

12. Program Throughput
Execution Time of pseudoJBB w/ 77MB Heap
600
GenCopy
500 GenMS
Execution Time (in s)
400
300
200
100
0
212 163 153 143 133 123 113 103 93
Available Memory
Paging causes a 40 to 62-fold increase in runtime
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

13. Outline
Motivation
GC without paging
Cooperative garbage collection
Bookmarking
Results
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

14. The problem
Garbage collector: VM-oblivious
Examines all reachable objects
Lacks knowledge of page residency
Treats evicted and resident pages identically
Virtual memory: GC-oblivious
GC & application have different access
patterns
Likely to evict pages needed by GC
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

15. Cooperative Garbage Collection
Bookmarking Extended virtual
collector memory manager
page eviction
notification
Select victim(s)
Page
Update
replacement
victim page(s)
residency
In response to notifications, BC:
Adjusts heap size to fit in main memory
Prevents eviction of important pages
Avoids touching non-resident pages
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

16. Avoiding Page Evictions
0 0 0 0 0
RAM
Hard Disk
When notified, avoid a pending eviction…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

17. Avoiding Page Evictions
0 0 0 0 0
RAM
Hard Disk
…find a page BC knows to be empty…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

18. Avoiding Page Evictions
0 0 0 0 0
RAM
Hard Disk
… and discard it…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

19. Avoiding Page Evictions
0 0 0 0
RAM
Hard Disk
… eliminating the need to evict a page.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

20. Limit of Heap Sizing
Could collect, compact, compress, etc.
Eventually:
Will run out of pages to discard
Going to have to evict non-empty pages
Result: Paging
Can we avoid this?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

21. Bookmarks
We introduce bookmarks:
Summaries of connectivity info on
evicted pages
References from objects on the page
These summaries enable GC w/o paging
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

22. Bookmarking
0 0 0 0
RAM
Hard Disk
Process page before eviction…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

23. Bookmarking
0 0 0 0
B
B
RAM
Hard Disk
... by following pointers & bookmark-ing targets...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

24. Bookmarking
1
0 0 0 0
B
B
RAM
Hard Disk
... increment the referring page count...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

25. Bookmarking
0 0 1 0
B B
B B B
B B
RAM
Hard Disk
... conservatively bookmark objects on the page...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

26. Bookmarking
1
0 0 0
B
B
RAM
0
B B
B B
B
Hard Disk
... then tell extended VM to evict the page.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

27. Bookmarking Details
Cheap summary of connectivity
One bit per object: free
One word per page: referring page count
Bookmarks cleared when count = zero
Use bookmarks as secondary roots
during garbage collection
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

28. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
Process objects as usual, but...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

29. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
... ignore any references to evicted pages.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

30. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
Use bookmarks to recreate evicted references...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

31. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
... and continue collection.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

32. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
Result: Garbage collection without paging!
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

33. Collection with Bookmarks
0 1 0
roots
B
B
RAM
0
B B
B B
B
Hard Disk
Note: can waste space on evicted pages.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

34. Bookmarking Incompleteness
Space waste not just on evicted pages
Collection with bookmarks is
necessarily incomplete
Not guaranteed to reclaim all memory
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

35. Bookmarking Incompleteness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
When a reference to an evicted object changes…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

36. Bookmarking Incompleteness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
When a reference to an evicted object changes…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

37. Bookmarking Incompleteness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
…it can make evicted objects unreachable.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

38. Bookmarking Incompleteness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
But bookmarks cannot be removed…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

39. Bookmarking Incompleteness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
…retaining unreachable heap objects.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

40. Bookmarking Completeness
Worst-case: completeness requires
duplicating evicted pages
See paper for more info
How can we preserve completeness?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

41. Bookmarking Completeness
0 1 0
B
B
RAM
0
B B
B B
B
Hard Disk
If the heap becomes full…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

42. Bookmarking Completeness
0 0 0
RAM
0
Hard Disk
…BC removes all bookmarks…
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

43. Bookmarking Completeness
0 0 0
???
0
???
…and performs a VM-oblivious collection...
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

44. Bookmarking Completeness
0 0 0
???
0
???
…reclaiming all unreachable objects.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

45. Bookmarking Completeness
0 0 0
???
BC’s worst case is other
collectors’ common case
0
???
…reclaiming all unreachable objects.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

46. BC Performance Optimizations
Uses generational design similar to
GenMS
Yields good performance when not
paging
Compacts heap to reduce pressure
Prevents single object from tying down a
page
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

47. Outline
Motivation
GC without paging
Cooperative garbage collection
Bookmarking
Results
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

48. Experimental Methodology
Extended Linux kernel 2.4.20
Eviction notification
vm_relinquish()
Added only 600 LOC
Jikes RVM 2.3.2 & MMTk
Compare BC to MarkSweep, SemiSpace,
CopyMS, GenCopy, GenMS
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

49. Throughput w/o Memory Pressure
Geo. Mean for Execution Time Relative to BC
1.15
BC
GenCopy
GenMS
Relative Execution Time
1.10
1.05
1.00
0.95
1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 3.75 4.00
Relative Heap Size
BC runtime comparable to GenMS
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

50. Throughput while Paging
Execution Time for pseudoJBB w/ 77MB Heap
600
Ideal
BC
500 GenCopy
Execution Time (in s)
GenMS
400
300
200
100
0
212 163 153 143 133 123 113 103 93
Available Memory
BC throughput closely follows ideal curve
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

51. BMU Curve w/ Memory Pressure
71.9%
12231 ms
39.1%
467944 ms
0.0% 0.0%
1566 ms 81591 ms
Bookmarking crucial for good performance
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

52. Summary of Results
When not paging:
BC as fast as GenMS
When paging:
vs. GenMS (fastest when not paging)
41x faster, avg pause up to 218x smaller
vs. CopyMS (next fastest when paging)
5x faster, avg pause up to 45x smaller
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

53. Conclusion
Bookmarking collector available at:
http://www.cs.umass.edu/~hertz
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

54. Thank you
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

55. Other Pointer Summarizations
Pointer Compression
Save space by compressing pointers
Tracing pointers becomes expensive
Remembered Sets
Add evicted pointers to buffers
Requires space upon pages eviction
Per Object Referring Page Counts
Increases BC overheads
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

56. Related Work
VM-Sensitive Garbage Collection
Linearizing LISP lists [Bobrow/Murphy]
Does not know or use which heap pages are
memory resident
Independent heap regions [Bishop]
Cannot avoid page faults when region
contains evicted pages
Ephemeral garbage collection [Moon]
Must access evicted pages when they contain
pointers into generations being collected
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

57. Related Work
VM-Cooperative Garbage Collection
Discarding empty pages [Cooper, et al.]
No mechanism for evicting non-empty pages
Shrinking heap size [Alonso/Appel]
Responds to memory pressure changes only
after a collection
Automatic heap sizing [Yang, et al.]
Orthogonal approach that determines proper
heap size
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

58. Throughput w/ Memory Pressure
Execution Time for pseudoJBB w/ 77MB Heap
500000
BC
450000
BC w/ Fixed Nursery
BC w/ Fixed Nursery
400000
and Resizing Only
Execution Time (ms)
GenCopy w/ Fixed
350000
Nursery
GenMS w/ Fixed
300000
Nursery
250000
200000
150000
100000
50000
0
212 163 153 143 133 123 113 103 93
Available Memory
BC outperforms fixed nursery collectors
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

59. Multiprogramming Performance
Elapsed Time for 2 runs of pseudoJBB, 77MB Heap
500000
BC
GenCopy
Total Wall Clock Time (ms)
GenMS
400000 CopyMS
SemiSpace
300000
200000
100000
0
491 292 282 272 262 252 242 232 222 212 202 192 182 172 162
Available Memory (MB)
BC performs well in many environments
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

60. Experimental Methodology
Benchmarks:
SPECjvm98, ipsixql, jython, and
pseudoJBB
“Opt-and-reset” methodology
First iteration optimizes all code
Record results from second run
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science