Managed runtime performance expert, Monica Beckwith will divulge her survival guide which is essential for any application performance engineer. Following simple rules and performance engineering patterns will make you and your stakeholders happy.

2. About Me

3. Performance
Engineering

4. ©2016 CodeKaram3

5. ©2016 CodeKaram4

6. Performance
Requirements

7. AskQuestions
©2016 CodeKaram6
• What makes the users happy?
• What makes them unhappy?
• Any current issues that need to be tackled?

8. Define and Understand
©2016 CodeKaram7
• Understand QoS (Quality of Service) for end users
• Define QoS success metrics in measurableterms
• Those are the service level agreements (SLAs)!

9. ©2016 CodeKaram8
• Document, understand and prioritize SLAs
• Throughput
• Responsetime
• Capacity
• Footprint
• CPU utilization?
• …
Define Success!

10. ©2016 CodeKaram9
Monitor, measure and define performance in
terms of throughput, latency, capacity,
footprint, utilization …
Work For It!

11. Defining Success!

12. ©2016 CodeKaram11

13. ©2016 CodeKaram12

14. Throughput
©2016 CodeKaram13
• Expected throughput?
• Can you fall below the expected throughput?
• How long can you stay below the expected throughput?
• What is the lowest that you can go?

15. Throughput
©2016 CodeKaram14
• Throughput measurement
• How is it measured?
• Transactions/sec; messages/sec or all of them?
• Where is it measured?
• Client-side; server-side; browser?

16. ©2016 CodeKaram15

17. ©2016 CodeKaram16

18. ResponseTime
©2016 CodeKaram17
• Expected response time?
• Can you go above the expected response time?
• How long can you stay there?
• How much can you exceed?

19. ResponseTime
©2016 CodeKaram18
• Response time measurement
• How is it measured?
• 99th percentile; 5-9s; worst-case; or all?
• Where is it measured?
• Client-side; server-side? Complete loop?

20. ©2016 CodeKaram19

21. ©2016 CodeKaram20

22. ©2016 CodeKaram21

23. Capacity Management
©2016 CodeKaram22
• What is the expected load?
• What happens if one system gets loaded more than others?
(load balancer issue)
• How is it measured?

24. Capacity Management
©2016 CodeKaram23
• What’s the max load that a system and all systems can tolerate?
• How long?
• What metrics are being captured?

25. Performance Analysis

26. Performance Analysis
^

27. Performance Analysis
©2016 CodeKaram26
• Analyze what factors enable the end-user experience to
meet or exceed the promised QoS
• Track your SLAs!

28. Java Application Stack
©2016 CodeKaram27
OS
Hardware
Application Ecosystem
Application
JRE
JVM

29. ©2016 CodeKaram28
• Application services
• Application server
• Database
• Any other services in the ecosystem?
Application Performance Analysis
Application Ecosystem
Application

30. JVM + Runtime Performance Analysis
©2016 CodeKaram29
JRE
JVM
• Classloading stats
• JIT Compilation stats
• Garbage Collection stats
• Threads stats

31. OS Performance Analysis
©2016 CodeKaram30
• System/ Kernel stats
• Lock stats
• Threads stats
OS

32. Hardware Performance Analysis
©2016 CodeKaram31
• Memory bandwidth/ traffic/ consumption
• CPU/ core utilization
• CPU cache efficiency/ utilization/ levels
• Architectural specific?
• IO Stats
Hardware

33. What areYouTrying to Achieve?
©2016 CodeKaram32
Improve application?

34. Top-Down Approach
©2016 CodeKaram33
OS
Hardware
Application Ecosystem
Application
JRE
JVM

35. Top Down Approach - Process
•Step 1Monitor
•Step 2 Profile
•Step 3Analyze
•Step 4 Tune +
Apply
©2016 CodeKaram34

36. What areYouTrying to Achieve?
©2016 CodeKaram35
Improve the platform?

37. Bottom-Up Approach
©2016 CodeKaram36
OS
Hardware
Application Ecosystem
Application
JRE
JVM

38. Bottom Up Approach - Process
•Step 1Monitor
•Step 2 Profile
•Step 3Analyze
•Step 4 Tune +
Apply
©2016 CodeKaram37

39. Top-Down Approach

40. Top-Down Approach
©2016 CodeKaram39
I HAVE the power!!
… to modify the code

41. Top-Down Approach
©2016 CodeKaram40
OS
Hardware
Application Ecosystem
Application
JRE
JVM

42. Top-Down Approach
©2016 CodeKaram41
Application Ecosystem
Application
OS
Hardware
JRE
JVM
Platform

43. Step 1: Monitor
©2016 CodeKaram42
Monitor and plot SUT (System UnderTest) statistics

44. ©2016 CodeKaram43
Step 1: Monitor

45. ©2016 CodeKaram44
Step 1: Monitor

46. ©2016 CodeKaram45
Step 1: Monitor

47. ©2016 CodeKaram46
Step 1: Monitor

48. ©2016 CodeKaram47
Step 1: Monitor

49. ©2016 CodeKaram48
• Tools?
• VisualVM, Java Flight Recorder
• PrintCompilation, PrintGCDetails (+PrintGCDateStamps), jmap –
clstats, jcmd GC.class_stats
Step 1: Monitor

50. ©2016 CodeKaram49
• Tools?
• Linux – mpstat, sysstat – iostat,pidstat…, prstat, vmstat, dash, CPU-Z,
cacti …
• Windows – Performance Monitor,Task Manager, Resource Monitor,
CPU-Z, cacti ...
Step 1: Monitor

51. Step 2+3: Profile + Analyze
©2016 CodeKaram50
• You have all the data that you need!
• Identify areas of improvement
• Profile those potential hotspots
• Analyze those hotspots

52. ©2016 CodeKaram51
• Tools? (Free/Open source/GPL/BSD)
• Oracle Solaris Studio PerformanceAnalyzer, perf tools, PAPI,CodeXL,
Dtrace, Oprofile, gprof, LTT (linux trace toolkit)
• Java Application – VisualVM, Netbeans profiler, jconsole …
Step 2+3: Profile + Analyze

53. Step 4:Tune
©2016 CodeKaram52
• Tune the JVM/GC – select the right heap, the right
GC algorithm
• Age objects appropriately
• Promote only long-lived objects
• GC worker threads perVM (for stop-the-world
GC events)
• GC concurrent worker threads perVM

54. Step 4:Tune
©2016 CodeKaram53
• Tune the JVM/GC – select the right heap, the right
GC algorithm
• See if compressed oops can be helpful
• Larger heaps may need AlwaysPretouch to be
enabled and also UseLargePages of
appropriate size

55. Step 4:Tune
©2016 CodeKaram54
• Tune your code to meet or exceed your SLAs
• Appropriate ramp-ups and ramp-downs
• Object age and retention strategies
• Understand what forms your LDS (live data
set)
• Confirm you are measuring the right thing! J

56. Bottom-Up Approach

57. Bottom Up Approach
©2016 CodeKaram56
I NEED the power!!
… to stress the platform

58. Bottom-Up Approach
©2016 CodeKaram57
OS
Hardware
Application Ecosystem
Application
JRE
JVM

59. Bottom-Up Approach
©2016 CodeKaram58
Application Ecosystem
Application
OS
Hardware
JRE
JVM
Platform

60. Where to Start?
©2016 CodeKaram59
• Know what you are stressing
• Get/ write the appropriate workload/ application
• Get/write the appropriate tools

61. Know WhatYou Are Stressing!
©2016 CodeKaram60
CPU –
Gather performance counter information for your
CPU stats, core stats, cache hits, misses and levels,
branch predictions, pipeline information, order-of-
execution, load-store unit load and queues,etc

62. Know WhatYou Are Stressing!
©2016 CodeKaram61
Memory –
Gather performance counter information for
memory utilization, memorybandwidth, read-write
stats, max read bandwidth, max write bandwidth,
max cross traffic bandwidth, architectural related
considerations, max capacity, etc

63. Know WhatYou Are Stressing!
©2016 CodeKaram62
JVM / GC–
Gather information related to the change – e.g. new
GC!
Gather information on different GC phases, parallel
work queues and work performance, concurrent
work and pressure,internal queues and buffers, any
GC work that’s staged?, etc

64. Where to Next?
©2016 CodeKaram63
• Know what you are stressing
• Get/ write the appropriate workload/ application
• Get/write the appropriate tools

65. Let’s have some fun!

66. Further Reading

68. www.codekaram.com
www.slideshare.net/monicabeckwith