In this presentation, Azul CTO Gil Tene describes why many Java warm-up strategies don't work, and how to slash warm-up time without rearchitecting. Learn why your warm-up strategies don't always work, how Java trading systems can start fast and stay fast, how to save and re-use optimizations across runs and how new JVM innovations can help you win.

1. ©2013 Azul Systems, Inc.
Winning with Java
At Market Open
Getting Fast & Staying Fast
Gil Tene, CTO & co-Founder, Azul Systems

2. ©2013 Azul Systems, Inc.
Are you fast at Market Open?

3. ©2013 Azul Systems, Inc.
Java at Market Open
. . .
Market Open

4. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality

5. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast

6. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization

7. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case

8. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt

9. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt

10. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
. . .

11. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
Warmup
. . .

12. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
Warmup
Deoptimization
. . .

13. Speculative compiler tricks
JIT compilers can do things that
static compilers can have
a hard time with…

14. ©2012 Azul Systems, Inc.
Untaken path example

15. ©2012 Azul Systems, Inc.
Untaken path example
“Never taken” paths can be optimized away with benefits:
void computeMagnitude(int val) {
if (val > 10) {
bias = computeBias(val);
else {
bias = 1;
}
return Math.log10(bias + 99);
}

16. ©2012 Azul Systems, Inc.
Untaken path example
“Never taken” paths can be optimized away with benefits:
void computeMagnitude(int val) {
if (val > 10) {
bias = computeBias(val);
else {
bias = 1;
}
return Math.log10(bias + 99);
}
When all values so far were <= 10 , can be optimized to:
void computeMagnitude(int val) {
if (val > 10) uncommonTrap();
return 2;
}

17. ©2012 Azul Systems, Inc.
Implicit Null Check example

18. ©2012 Azul Systems, Inc.
Implicit Null Check example
All field and array access in Java is null checked
x = foo.x;
is (in equivalent required machine code):
if (foo == null)
throw new NullPointerException();
x = foo.x;

19. ©2012 Azul Systems, Inc.
Implicit Null Check example
All field and array access in Java is null checked
x = foo.x;
is (in equivalent required machine code):
if (foo == null)
throw new NullPointerException();
x = foo.x;
But compiler can “hope” for non-nulls, and handle SEGV
<Point where later SEGV will appear to throw>
x = foo.x;

20. ©2012 Azul Systems, Inc.
Implicit Null Check example
All field and array access in Java is null checked
x = foo.x;
is (in equivalent required machine code):
if (foo == null)
throw new NullPointerException();
x = foo.x;
But compiler can “hope” for non-nulls, and handle SEGV
<Point where later SEGV will appear to throw>
x = foo.x;
This is faster *IF* no nulls are encountered…

21. Deoptimization

22. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive

23. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative

24. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior

25. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong

26. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong
Warming up code is a black art

27. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong
Warming up code is a black art
Running code long enough to compile is just the start…

28. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong
Warming up code is a black art
Running code long enough to compile is just the start…
Deoptimization can occur at any time

29. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong
Warming up code is a black art
Running code long enough to compile is just the start…
Deoptimization can occur at any time
often occur after you *think* the code is warmed up.

30. ©2012 Azul Systems, Inc.
Deoptimization:
Adaptive compilation is… adaptive
JIT Compilers are adaptive and speculative
Leverage observations of behavior
Can “hope” about things and can recover if wrong
Warming up code is a black art
Running code long enough to compile is just the start…
Deoptimization can occur at any time
often occur after you *think* the code is warmed up.
Deoptimization has many potential causes

31. ©2012 Azul Systems, Inc.
Warmup often doesn’t cut it…

32. ©2012 Azul Systems, Inc.
Warmup often doesn’t cut it…
Common Example:
Trading system wants to have the first trade be fast
So run 20,000 “fake” messages through the system to warm up
let JIT compilers optimize, learn, and deopt before actual trades

33. ©2012 Azul Systems, Inc.
Warmup often doesn’t cut it…
Common Example:
Trading system wants to have the first trade be fast
So run 20,000 “fake” messages through the system to warm up
let JIT compilers optimize, learn, and deopt before actual trades
What really happens
Code is written to do different things “if this is a fake message”
e.g. “Don’t send to the exchange if this is a fake message”
JITs optimize for fake path, including speculatively assuming “fake”
First real message through deopts...

34. ©2012 Azul Systems, Inc.
A few example causes for
depotimization

35. ©2012 Azul Systems, Inc.
A few example causes for
depotimization
Newly encountered code breaking prior assumptions

36. ©2012 Azul Systems, Inc.
A few example causes for
depotimization
Newly encountered code breaking prior assumptions
First calls to method in an uninitialized class

37. ©2012 Azul Systems, Inc.
A few example causes for
depotimization
Newly encountered code breaking prior assumptions
First calls to method in an uninitialized class
First call to a method in a not-yet-loaded class

38. ©2012 Azul Systems, Inc.
A few example causes for
depotimization
Newly encountered code breaking prior assumptions
First calls to method in an uninitialized class
First call to a method in a not-yet-loaded class
Dynamic branch elimination hitting an unexpected path

39. ©2012 Azul Systems, Inc.
A few example causes for
depotimization
Newly encountered code breaking prior assumptions
First calls to method in an uninitialized class
First call to a method in a not-yet-loaded class
Dynamic branch elimination hitting an unexpected path
Implicit Null Check hitting a null

40. ©2013 Azul Systems, Inc.
Java’s “Just In Time” Reality
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
Warmup
Deoptimization
. . .

41. ©2013 Azul Systems, Inc.
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
What we have
Java’s “Just In Time” Reality

42. ©2013 Azul Systems, Inc.
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
What we have What we want
Java’s “Just In Time” Reality

43. ©2013 Azul Systems, Inc.
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
What we have What we want
No Slow Ops or Trades
Java’s “Just In Time” Reality

44. ©2013 Azul Systems, Inc.
Starts slow, learns fast
Lazy loading & initialization
Aggressively optimized for
the common case
(temporarily) Reverts to
slower execution to adapt
What we have What we want
No Slow Ops or Trades
ReadyNow!
to the rescue
Java’s “Just In Time” Reality

45. ©2012 Azul Systems, Inc.
What can we do about it?

46. ©2012 Azul Systems, Inc.
What can we do about it?
Azul’s Zing JVM has a feature set called “ReadyNow!”
Avoids deoptimization while keeping code fast
Allows code to reach optimized levels without requiring “exercise”

47. ©2012 Azul Systems, Inc.
What can we do about it?
Azul’s Zing JVM has a feature set called “ReadyNow!”
Avoids deoptimization while keeping code fast
Allows code to reach optimized levels without requiring “exercise”
Learning and retaining optimizations
With Zing, your next run can learn from the previous one

48. ©2012 Azul Systems, Inc.
What can we do about it?
Azul’s Zing JVM has a feature set called “ReadyNow!”
Avoids deoptimization while keeping code fast
Allows code to reach optimized levels without requiring “exercise”
Learning and retaining optimizations
With Zing, your next run can learn from the previous one
Fine grained control options
Per method compiler directives
…

49. ©2012 Azul Systems, Inc.
Logging & “replaying”optimizations

50. ©2012 Azul Systems, Inc.
Logging & “replaying”optimizations
Zing’s ReadyNow includes optimization logging
Records ongoing optimization decisions and stats
records optimization dependencies
Establishes “stable optimization state” at end of previous run

51. ©2012 Azul Systems, Inc.
Logging & “replaying”optimizations
Zing’s ReadyNow includes optimization logging
Records ongoing optimization decisions and stats
records optimization dependencies
Establishes “stable optimization state” at end of previous run
Zing can then read prior logs at startup
Prime JVM with knowledge of prior stable optimizations
Optimizations are applied as their dependencies get resolved

52. ©2012 Azul Systems, Inc.
Logging & “replaying”optimizations
Zing’s ReadyNow includes optimization logging
Records ongoing optimization decisions and stats
records optimization dependencies
Establishes “stable optimization state” at end of previous run
Zing can then read prior logs at startup
Prime JVM with knowledge of prior stable optimizations
Optimizations are applied as their dependencies get resolved
ReadyNow workflow promotes confidence
You’ll know if/when all optimizations have been applied
If some optimization haven’t been applied, you’ll know why…

53. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start

54. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start
Deoptimization

55. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start
Deoptimization
ReadyNow!
avoids
deoptimization

56. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start
With Zing & ReadyNow!

57. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start
With ReadyNow!
Warmup?

58. ©2013 Azul Systems, Inc.
Java at “Load Start”
. . .
Load Start
With ReadyNow!
Warmup?
Fast From
The Start

59. ©2013 Azul Systems, Inc.
. . .
Load Start
With ReadyNow! and
Start Fast & Stay Fast
Java at “Load Start”

60. ©2013 Azul Systems, Inc.
. . .
Load Start
With Zing & ReadyNow!
Start Fast & Stay Fast
Java at “Load Start”

61. ©2013 Azul Systems, Inc.
. . .
Load Start
With Zing & ReadyNow!
Start Fast & Stay Fast
Java at “Load Start”

62. ©2013 Azul Systems, Inc.
. . .
Load Start
With Zing & ReadyNow!
Start Fast & Stay Fast
Java at “Load Start”
With
Confidence

63. ©2013 Azul Systems, Inc.
One liner takeaway
. . .
Load Start

64. ©2013 Azul Systems, Inc.
One liner takeaway
. . .
Zing: A cure for the Java hiccups
Load Start