Gclogs

2. Kodewerk About Me tm Java Performance Services • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved

3. Kodewerk About Me tm Java Performance Services • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved

4. Kodewerk Disclaimer tm Java Performance Services The resemblance of any opinion, recommendation or comment made during this presentation to performance tuning advice is merely coincidental. Copyright 2011 Kodewerk Ltd. All rights reserved

6. hy G C d at be o Kodewerk fo tha we g in is tm Java Performance Services re t oc ha no nn g th cu ve in e tting a heap ru I am ge en rs th th ju is an is or no apparent d OOM f of st lo ha 0% PU d Hi! can W hat . reasonc d zen. C g the o uld ay in stra size be t h he why GC’s e en w f re d u r in e rea h th feT gc som son f gc.l it h g s ta pp 10 nge r tu p or co que ntinu heap CM t f ro exe m w ous f r a . Wi t og. eon ull , I Wh ntly S G cut y afte oble fr do It s e h mca t im e ance wan ee es C e es? se ze ou and elp co up eem r a and a pr t to es to nd xec fo or m pp e of s? r 1 lic to why 0s at s v Hello ute io rows kno of I’ve perf inut n see laps er y s so Copyright 2011 Kodewerk Ltd. All rights reserved fu l w

9. Kodewerk tm Garbage Collection Logs Java Performance Services -verbose:gc -Xloggc:gc.log 7.0 has new ﬂags for rollover Some other useful ﬂags -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintHeapAtGC -XX:+PrintApplicationStoppedTime Copyright 2011 Kodewerk Ltd. All rights reserved

10. Kodewerk tm Collector Choices Java Performance Services uint i = 0; if (UseSerialGC) i++; if (UseConcMarkSweepGC || UseParNewGC) i++; if (UseParallelGC || UseParallelOldGC) i++; if (UseG1GC) i++; if (i > 1) { jio_fprintf(defaultStream::error_stream(), "Conflicting collector combinations in option list; " "please refer to the release notes for the combinations " "allowedn"); status = false; } Copyright 2011 Kodewerk Ltd. All rights reserved

11. Kodewerk tm Java Memory Pools Java Performance Services Eden S0 S1 Tenured Perm Young Old Gen Gen Normal Flow - objects created in Eden - copied into the active survivor space - copied between survivor spaces - copied to Old Gen when ages reaches tenuring threshold Copyright 2011 Kodewerk Ltd. All rights reserved

12. Kodewerk tm Java Memory Pools Java Performance Services Eden S0 S1 Tenured Perm Young Old Gen Gen Abnormal Situations - memory leaks (loitering objects) - long pause times (heap too big, poor parallelization) - too frequent full GC (zombies, poor throughput) - survivor space flooding (premature promotion) - fragmented tenured space (compaction) - high rates of object creation - etc..... Copyright 2011 Kodewerk Ltd. All rights reserved

14. Kodewerk tm Java Performance Services Puzzler #1 600 Heap Usage after GC (MB) 500 400 300 what happened here? 200 100 1000 2000 3000 4000 Time (seconds) Copyright 2011 Kodewerk Ltd. All rights reserved

15. Kodewerk tm GC Log Fragment Java Performance Services 21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K- >89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02, real=0.32 secs] 163.028: [GC 163.028: [DefNew Desired survivor size 115671040 bytes, new threshold 15 (max 15) - age 1: 736 bytes, 736 total - age 2: 112 bytes, 848 total - age 4: 40 bytes, 888 total - age 6: 232 bytes, 1120 total - age 7: 32 bytes, 1152 total - age 11: 96 bytes, 1248 total : 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] Heap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000) No shared spaces conﬁgured. Copyright 2011 Kodewerk Ltd. All rights reserved

16. Kodewerk tm Java Performance Services Full GC 21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K- >89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02, real=0.32 secs] Full GC requested @, started @ Copyright 2011 Kodewerk Ltd. All rights reserved

17. Kodewerk tm Java Performance Services Full GC 21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K- >89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02, real=0.32 secs] tenured space statistics perm space statistics - other stats are for the entire heap Copyright 2011 Kodewerk Ltd. All rights reserved

18. Kodewerk tm Java Performance Services Full GC 21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K- >89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02, real=0.32 secs] - occupancy before->after collection (memory pool conﬁgured size) - reporting on tenured, perm memory pools plus total for all Java heap spaces Copyright 2011 Kodewerk Ltd. All rights reserved

19. Kodewerk tm Java Performance Services Full GC 21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K- >89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02, real=0.32 secs] - pause time statistics for tenured and all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved

20. Kodewerk tm GC Log Fragment Java Performance Services 163.028: [GC 163.028: [DefNew Desired survivor size 115671040 bytes, new threshold 15 (max 15) - age 1: 736 bytes, 736 total - age 2: 112 bytes, 848 total - age 4: 40 bytes, 888 total - age 6: 232 bytes, 1120 total - age 7: 32 bytes, 1152 total - age 11: 96 bytes, 1248 total : 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] young gen statistics statistics for all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved

21. Kodewerk tm GC Log Fragment Java Performance Services 163.028: [GC 163.028: [DefNew Desired survivor size 115671040 bytes, new threshold 15 (max 15) - age 1: 736 bytes, 736 total - age 2: 112 bytes, 848 total - age 4: 40 bytes, 888 total - age 6: 232 bytes, 1120 total - age 7: 32 bytes, 1152 total - age 11: 96 bytes, 1248 total : 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] surivor space sizing and tenuring thresholds breakdown of bytes @ age Copyright 2011 Kodewerk Ltd. All rights reserved

22. Kodewerk tm GC Log Fragment Java Performance Services Heap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000) No shared spaces conﬁgured. - breakdown of occupancy and conﬁgured size of each memory pool - pointer values describing each space (for debugging) Copyright 2011 Kodewerk Ltd. All rights reserved

23. Kodewerk tm Java Performance Services Puzzler #2 0.194: [GC [PSYoungGen: 10489K->720K(305856K)] 10489K->720K(1004928K), 0.0061190 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 0.200: [Full GC (System) [PSYoungGen: 720K->0K(305856K)] [ParOldGen: 0K->594K(699072K)] 720K->594K(1004928K) [PSPermGen: 6645K- >6641K(204800K)], 0.0516540 secs] [Times: user=0.10 sys=0.00, real=0.06 secs] 6.184: [GC [PSYoungGen: 262208K->14797K(305856K)] 262802K->15392K(1004928K), 0.0354510 secs] [Times: user=0.18 sys=0.04, real=0.03 secs] 9.549: [GC [PSYoungGen: 277005K->43625K(305856K)] 277600K->60736K(1004928K), 0.0781960 secs] [Times: user=0.56 sys=0.07, real=0.08 secs] 11.768: [GC [PSYoungGen: 305833K->43645K(305856K)] 322944K->67436K(1004928K), 0.0584750 secs] [Times: user=0.40 sys=0.05, real=0.06 secs] 15.037: [GC [PSYoungGen: 305853K->43619K(305856K)] 329644K->72932K(1004928K), 0.0688340 secs] [Times: user=0.42 sys=0.01, real=0.07 secs] 19.372: [GC [PSYoungGen: 273171K->43621K(305856K)] 302483K->76957K(1004928K), 0.0573890 secs] [Times: user=0.41 sys=0.01, real=0.06 secs] 19.430: [Full GC (System) [PSYoungGen: 43621K->0K(305856K)] [ParOldGen: 33336K->54668K(699072K)] 76957K->54668K(1004928K) [PSPermGen: 36356K->36296K(204800K)], 0.4569500 secs] [Times: user=1.77 sys=0.02, real=0.46 secs] 19.924: [GC [PSYoungGen: 4280K->128K(305856K)] 58949K->54796K(1004928K), 0.0041070 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 19.928: [Full GC (System) [PSYoungGen: 128K->0K(305856K)] [ParOldGen: 54668K->54532K(699072K)] 54796K->54532K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.3531480 secs] [Times: user=1.19 sys=0.10, real=0.35 secs] 20.284: [GC [PSYoungGen: 4280K->64K(305856K)] 58813K->54596K(1004928K), 0.0040580 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 20.288: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54532K->54532K(699072K)] 54596K->54532K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.2360580 secs] [Times: user=1.01 sys=0.01, real=0.24 secs] 20.525: [GC [PSYoungGen: 4280K->96K(305856K)] 58813K->54628K(1004928K), 0.0030960 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 20.528: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54532K->54533K(699072K)] 54628K->54533K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.2311320 secs] [Times: user=0.88 sys=0.00, real=0.23 secs] 20.760: [GC [PSYoungGen: 4280K->96K(305856K)] 58814K->54629K(1004928K), 0.0034940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 20.764: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54533K->54533K(699072K)] 54629K->54533K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.2381600 secs] [Times: user=0.85 sys=0.01, real=0.24 secs] 21.201: [GC [PSYoungGen: 5160K->354K(305856K)] 59694K->54888K(1004928K), 0.0019950 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 21.204: [Full GC (System) [PSYoungGen: 354K->0K(305856K)] [ParOldGen: 54533K->54792K(699072K)] 54888K->54792K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.2358570 secs] [Times: user=0.98 sys=0.01, real=0.24 secs] 21.442: [GC [PSYoungGen: 4280K->64K(305856K)] 59073K->54856K(1004928K), 0.0022190 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 21.444: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54792K->54792K(699072K)] 54856K->54792K(1004928K) [PSPermGen: 36300K->36300K(204800K)], 0.2475970 secs] [Times: user=0.95 sys=0.00, real=0.24 secs] 21.773: [GC [PSYoungGen: 11200K->741K(305856K)] 65993K->55534K(1004928K), 0.0030230 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 21.776: [Full GC (System) [PSYoungGen: 741K->0K(305856K)] [ParOldGen: 54792K->54376K(699072K)] 55534K->54376K(1004928K) [PSPermGen: 36538K->36537K(204800K)], 0.2550630 secs] [Times: user=1.05 sys=0.00, real=0.25 secs] Copyright 2011 Kodewerk Ltd. All rights reserved

26. Kodewerk tm Java Performance Services Puzzler #3 848463.696: [GC [1 CMS-initial-mark: 989633K(1075200K)] 1003551K(1397760K), 6.3838946 secs] [Times: user=0.06 sys=0.01, real=6.39 secs] 848470.085: [CMS-concurrent-mark-start] 848605.512: [CMS-concurrent-mark: 135.377/135.426 secs] [Times: user=7.83 sys=9.92, real=135.44 secs] 848605.512: [CMS-concurrent-preclean-start] 848605.726: [CMS-concurrent-preclean: 0.207/0.214 secs] [Times: user=0.06 sys=0.00, real=0.22 secs] 848605.726: [CMS-concurrent-abortable-preclean-start] CMS: abort preclean due to time 848610.846: [CMS-concurrent-abortable-preclean: 1.014/5.120 secs] [Times: user=1.05 sys=0.08, real=5.13 secs] 848610.871: [GC[YG occupancy: 160577 K (322560 K)]848610.871: [Rescan (parallel) , 0.1280728 secs]848610.999: [weak refs processing, 1.3120556 secs] [1 CMS-remark: 989633K(1075200K)] 1150210K(1397760K), 1.4413516 secs] Application runtime details can help threads stall for 60 seconds during pause Copyright 2011 Kodewerk Ltd. All rights reserved

27. Kodewerk tm Java Performance Services Puzzler #4 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K- >62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K- >52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid18706.hprof ... ???? Heap dump ﬁle created [83874513 bytes in 3.330 secs] 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K- >76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved

28. Kodewerk tm Java Performance Services Puzzler #4 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K- >62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K- >52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid18706.hprof ... Heap dump ﬁle created [83874513 bytes in 3.330 secs] 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K- >76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved

30. Kodewerk tm Java Performance Services Puzzler #4 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K- >62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K- >52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid18706.hprof ... Heap dump ﬁle created [83874513 bytes in 3.330 secs] 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K- >76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] 147.823 + 0.8074580 = 148.630 ~= 148.632 Copyright 2011 Kodewerk Ltd. All rights reserved

32. Kodewerk tm Java Performance Services Puzzler #4 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K- >62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K- >52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid18706.hprof ... ???? Heap dump ﬁle created [83874513 bytes in 3.330 secs] 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K- >76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved

Gclogs

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Viewers also liked

Viewers also liked (8)

Similar to Gclogs

Similar to Gclogs (20)

Recently uploaded

Recently uploaded (20)

Gclogs

Editor's Notes