“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
•
0 recomendaciones•51 vistas
“Just leave the garbage outside and we will take care of it for you”. This is the panacea promised by garbage collection mechanisms built into most software stacks available today. So, we don’t need to think about it anymore, right? Wrong! When misused, garbage collectors can fail miserably. When this happens they slow down your application and lead to unacceptable pauses. In this talk we will go over different garbage collectors approaches and understand under which conditions they function well.
Recomendados
Recomendados
Más contenido relacionado
Similar a “Show Me the Garbage!”, Garbage Collection a Friend or a Foe
Similar a “Show Me the Garbage!”, Garbage Collection a Friend or a Foe (20)
Más de Haim Yadid
Más de Haim Yadid (19)
Último
Último (20)
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
- 1. Show Me the Garbage Haim Yadid @ Next Insurance
- 6. @lifeyx
- 7. NEXT 7 Today
- 8. NEXT 8 Talk Focus Mainstream applications HPC/ HFT/ Huge heaps Web apps latency Low latency Monitor and Tune no allocation
- 9. NEXT 9 Agenda ● Memory Management Terminology ● GC Building Blocks ● Current OpenJDK GCs ● Monitoring GC and Memory ● Analysis of common problems
- 12. NEXT Allocator - Memory Allocation mechanism Collector - memory reclamation mechanism 12 Participants
- 13. Mutator
- 14. NEXT Allocator - Memory Allocation mechanism Mutators - the application threads Collectors - the GC threads 14 Participants
- 15. NEXT ● new / malloc () to create object ● delete() / free() release an object 15 malloc / free
- 16. NEXT ● Dangling pointers / Double free ● Memory leaks ● Crashes ● Security Vulnerabilities ● Fragmentation ● Performance Degradation 16 I don’t miss it
- 17. NEXT 17 Automatic Reference Counting (ARC)
- 18. NEXT 1 1 Heap 2 1 1 1 0 Non Heap 18 Automatic Reference Counting (ARC)
- 19. NEXT 1 1 Heap 2 1 1 1 1 19 Automatic Reference Counting - Cycles
- 20. NEXT Concurrency - Every change to reference requires a memory barrier Very bad for highly concurrent systems Mostly relevant for single threaded 20 Reference Counting Concurrency
- 21. NEXT The (Tracing) Garbage Collector 21
- 22. NEXT 22
- 23. NEXT The 3 KPIs (key performance indicators) 23
- 24. NEXT CPU/Time Overhead The amount of time the GC consumes / requires ○ Wall clock time ○ CPU time 24
- 25. NEXT Memory Overhead The amount of memory the GC requires ○ Internal Data structures ○ Headroom 25
- 26. NEXT Latency (Pauses) What is the probability that the GC will affect your transaction time and for how long 26
- 28. NEXT Heap Static & Stack 28 To Heap or not to Heap
- 30. NEXT Heap Stack 30 Live Set (Reachable objects)
- 36. NEXT Heap 36 Root Scanning & Safe Points Work: O(#threads & stack frames)
- 37. NEXT Heap Work: O(#Objects in LiveSet) Stack 37 Marking (a.k.a. tracing)
- 38. NEXT Sweep through the heap and collect all unmarked objects Maintain a list of free memory portions Work: O(#Objects and fragments in Heap) Heap Unreachable Reachable Free 38 Mark & Sweep : Sweep
- 40. NEXT Moving objects to make bigger free contiguous spaces Heap Heap Work: O( Size of Liveset in Heap) 40 Mark Compact
- 41. NEXT 41 Moving object around ● fix incoming references ● Stop application ? ● Read and write memory barriers ○ Brook pointer (shenandoah) ○ colored pointers (zgc, C4)
- 42. NEXT Move all live objects to another space From Space To Space Work: O(Size of live set) Memory: x2 42 Copy collector
- 43. NEXT 43 Thread Local Bump allocation
- 44. NEXT Weak Generational Hypothesis - Most object die young The surviving ones tend to live for a long time 44 Generational collectors
- 45. NEXT Old Non Heap From To 45 New/ Old Generation there are also survivor spaces (I know)
- 46. NEXT Higher throughput Shorter new generation collection 46 Generational collectors
- 48. NEXT Single Threaded Stop the World Concurrent Parallel VS VS Monolithic Incremental VS 48 Collectors Attributes
- 49. NEXT 49 STW Collectors - Serial (single threaded) - Monolithic (Non incremental) stop the world Application Threads GC Thread
- 50. NEXT 50 STW Collectors - Incremental
- 51. NEXT 51 STW Collectors - Parallel Application Threads GC Threads
- 52. NEXT 52 Concurrent Collectors - (single threaded)
- 54. NEXT Let’s Talk about Mostly 54
- 55. NEXT When do we want to collect Garbage? 55
- 56. NEXT 56
- 57. NEXT 57 The Open JDK GCs - Serial Property Serial -XX:+UseSerialGC Compacting Regional Generational Concurrent Mostly Concurrent When To Use Small Heaps Single cpu containers (*) can be found as part of Corretto JDK
- 58. NEXT 58 The Open JDK GCs - Parallel Property Serial -XX:+UseSerialGC Parallel -XX:+UseParallelGC Compacting Regional Generational Concurrent Mostly Concurrent When To Use Small Heaps Single cpu containers High throughput Latency of a single response is not important (*) can be found as part of Corretto JDK
- 59. NEXT 59 The Open JDK GCs - G1GC Property Serial -XX:+UseSerialGC Parallel -XX:+UseParallelGC G1 -XX:+UseG1GC Compacting Regional Generational Concurrent <200ms Mostly Concurrent Old gen only When To Use Small Heaps Single cpu containers High throughput Latency of a single response is not important Mainstream applications (*) can be found as part of Corretto JDK
- 60. NEXT 60 The Open JDK GCs - ZGC Property Serial -XX:+UseSerialGC Parallel -XX:+UseParallelGC G1 -XX:+UseG1GC ZGC -XX:+UseG1GC Compacting Regional Generational WIP Concurrent <200ms <1ms Mostly Concurrent Old gen only When To Use Small Heaps Single cpu containers High throughput Latency of a single response is not important Mainstream applications Large Heaps Highly responsive (*) can be found as part of Corretto JDK
- 61. NEXT 61 The Open JDK GCs - Shenandoah Property Serial -XX:+UseSerialGC Parallel -XX:+UseParallelGC G1 -XX:+UseG1GC ZGC -XX:+UseG1GC Shenandoah(*) -XX:+UseShenandoahGC Compacting Regional Generational WIP WIP Concurrent <200ms <1ms <1ms Mostly Concurrent Old gen only When To Use Small Heaps Single cpu containers High throughput Latency of a single response is not important Mainstream applications Large Heaps Highly responsive Large Heaps Highly responsive (*) can be found as part of Corretto JDK
- 62. NEXT 62 Azul’s C4 Property C4 Compacting Regional Generational Concurrent Mostly Concurrent When To Use Large Heaps low latency
- 63. NEXT 63 Let’s go practical Dall-e: practical engineer picasso style
- 65. NEXT ● CPU/Time overhead ● Memory Overhead ● Latency and Pauses 65 So what should we monitor ?
- 66. NEXT -Xlog:gc=info: file=<file_name>: time,uptime,level,tags: filecount=5,filesize=20m 66 GC Logs Output file decorators Output Options Tag
- 67. NEXT 67 GC Logs [2022-09-01T17:30:40.433+0000][0.004s][info][gc] Using G1 [2022-09-01T17:30:41.975+0000][1.546s][info][gc] GC(0) Pause Young (Concurrent Start) (Metadata GC Threshold) 126M->30M(4096M) 12.598ms [2022-09-01T17:30:41.975+0000][1.546s][info][gc] GC(1) Concurrent Mark Cycle [2022-09-01T17:30:41.988+0000][1.558s][info][gc] GC(1) Pause Remark 32M->22M(4096M) 2.119ms [2022-09-01T17:30:41.988+0000][1.558s][info][gc] GC(1) Pause Cleanup 22M->22M(4096M) 0.003ms [2022-09-01T17:30:42.052+0000][1.622s][info][gc] GC(1) Concurrent Mark Cycle 76.584ms [2022-09-01T17:30:42.824+0000][2.395s][info][gc] GC(2) Pause Young (Concurrent Start) (Metadata GC
- 68. NEXT 68 GC Logs - Description [2022-09-01T17:32:31.262+0000][110.832s][info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms
- 69. NEXT 69 GC Logs - Time Stamp (when) [2022-09-01T17:32:31.262+0000][110.832s] [info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms
- 70. NEXT 70 GC Logs - Memory released (how much) [2022-09-01T17:32:31.262+0000][110.832s] [info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms
- 71. NEXT 71 GC Logs - Memory released (how long ) [2022-09-01T17:32:31.262+0000][110.832s] [info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms
- 72. NEXT 72 GC Logs [2022-09-01T17:32:31.262+0000][110.832s][info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms [2022-09-01T17:33:48.974+0000][188.545s][info][gc] GC(16) Pause Young (Normal) (G1 Evacuation Pause) 1240M->169M(4096M) 26.671ms . . . Start and End Time Freed memory Overhead Frequency Allocation Rate Pause Time
- 73. NEXT 73 GC Logs - GC Viewer
- 74. NEXT 74 GC Logs - GC Viewer
- 75. NEXT 75 GC Logs - GC Viewer
- 78. NEXT ● MBeans ● Mission Control 78 JMX (Java.lang.GarbageCollector)
- 79. NEXT 79 Visual GC * does not support ZGC and Shenandoah
- 80. NEXT ● Wake Up every x ms ● measure amount time passed ● https://github.com/giltene/jHiccup 80 Latency - Hiccup Probe (inspired by jHiccup) T T+x + p { Sleep X ms
- 81. NEXT ● GC logs and - System.gc() ● Histogram of live set : jmap -histo:live <PID> ● Grab a Heap dump (or at least a heap histogram) 81 Estimate Live Set
- 82. NEXT ● Long lasting memory ( maybe slowly changing) ● Service with large state will benefit from generational GC ● If State is bigger than old gen …. very bad 82 Live set - Long lasting - state
- 83. NEXT ● The (maximal/average) amount of memory needed to serve a request 83 Live set - Working memory
- 85. NEXT 85 https://github.com/lifey/gcKryptonites Pause Detector every 10ms detects (10& 100ms) State Holder Holds 2GB of smalls object (100m) Brutal Allocator Allocates 500 MB/s 2X
- 88. NEXT 88 Some Results(2 min execution 4GB heap) G1 Shenandoah zGC Parallel # 10ms pause 27 3 10 107 # 100 ms pause 0 1 4 9 Average runtime of iteration 302ms 794ms 352ms 434ms LiveSet 2GB 2GB 2.5GB 2GB
- 89. NEXT ● Is it over > 5% ● Improve by : ○ Increase heap size ○ Use Generational Garbage collector 89 Excess GC (Temporal) Overhead
- 90. NEXT ● What can you tolerate ? ● Move to a concurrent ● Upgrade to the newest JVM ● Use Zing JVM 90 Latency
- 91. NEXT ● Use Generational Garbage collector ● Trade throughput 91 Excess GC (Memory) Overhead
- 92. NEXT ● Compacting GC should not have fragmentation ● What about humongous objects ? ● G1 handles it poorly ● ZGC Does better 92 Fragmentation
- 93. NEXT ● long term state should fit in old gen ● short term memory should not spill to old gen 93 Generational GC anomalies New Generation Old Generation Application State
- 94. NEXT 94 Your Service Questionnaire: Question Value Heap size the process is using (-Xmx) GC overhead over an hour of peak time Max Pause Time over a 24 hour period Number of collections above GC pause time targets Live set size (MB) Allocation Rate