Slides from talk given on Java/Scala Lab 2014 in Odessa, Ukraine. Describes of how Java can be used as platform for latency-restricted applications such as High Frequency Trading and demonstrates how latencies 15-30µsec can be achieved on vanilla Oracle JDK.

1. Lynx Capital Partners
Use Java in HFT

2. Intro / Trading
•Brief history of financial markets
•Era of gesturing brokers/traders
•Electronic markets, machines replacing
humans
•Modern trading almost completely automated
(incl. crashes)

3. Information flows, briefly
Markets
Markets
Ticker plant
market data processing
marketdata, N x 10G
Historical data storage
Research clusterOrder stream
Execution management
Order/risk
management

4. Latencies matter

5. Nuances... (incl. the devil)
• There’s no such thing as current price
• Inter-market arbitrage, faster wins
• Good data stream is a thick data stream
• ... and it works being diligently processed (only)
• How thick? From GBs to TBs. Daily.
• ... ah, all that jazz is high-dimensional jazz
Complexity
chart
Development
costs

6. Alternatives (working ones)
• FPGA!
• Memory limitations...
• Algorithmic limitations...
• License and development costs...
• Scaling issues....
• Unmanaged native code (C++ doom)
• Long development cycle (debugging hell with imps)
• Error prone
• Complex business logic -> exponential costs growth
• May be Rust? But not now (yet)…

7. С++, eh?

8. Java! Familiar, isn’t it?
• Pros
• Scales better in regards to complexity
• Better performance at lower cost
• JIT produces the code of actual load
• ± the same result as native code if u know proper tricks
• Better interoperability with non-real time parts
• Easier deployment and configuration
• Scales well for bandwidth
• Rich tooling
• Cons
• Latency (damn GC, the sleeping dragon!)
• Lack of control of JIT’s optimiser

9. GC. Don’t wake the Dragon.

10. GC. Don’t wake the Dragon.
• #1 enemy
• Dying is forbidden
• At once we can call it to swipe garbage after
initialisation
• Preallocated and pooled, it’s how that all look alike
• Allocation trees can become vast and GC can climb
on them for ages, doing nothing
• Critical JNI can keep that bstrd behind the bars
• Frameworks? Didn’t hear that word
• Most of JDK utility classes are sinful
• Founding Fathers wrote their own, righteous utility classes

11. Direct Memory Access

12. Direct Memory Access
• Off-heap memory can be used
• Unsafe class to help allocate in OS heap
• Unsafe reads from and writes to direct addresses
• Manual memory management, finally! We kill them all
when we have ordered to do that.
• The framework to rule has been built
• No type safety and no range checks mean that strong
exorcism skills are required. Sometimes.
• Yes, it can blast the whole world off
• There’s a candy, though: better performance inside
NUMA group.

13. UnsafeUtil example

14. UnsafeUtil writing

15. UnsafeUtil reading

16. Unsafe: other pattern

17. Off-heap object allocation

18. Off-heap object allocation
• Everything fast is pricey
• Could we allocate Java objects off-heap?
• We developed the cure
• Pre-allocated mutable Java object which contents can be copied
off-heap creating new reference. Stairway to heaven.
• This “template” object never dies and saves all references to off-
heap copies
• There’s no direct access to heaven, and JVM is catcher in rye
• Setters have to be defined for all reference fields
• Dura lex, but there are tricks
• Transparent memory mapping w/o expl. serialization/parsing
• Pointer math (we missed it in Java!)
• Passing objects between threads

19. Off-heap object example

20. Off-heap object example

21. Off-heap object example

22. The art of context switching
• Random and significant delay
• from 500µs to 10s of ms - long as dark ages
• To prevent contextquakes threads are nailed
down to physical CPU cores
• Bless OS for system calls for thread affinity
• But OS still can suspend your thread
• So, OS is not your friend, no, bro
• Avoid any interaction with this troublemaker (OS)
• Including OS scheduler and shared IO

23. Thread blocking

24. Thread blocking
• LockSupport class allows allocation-free block/unblock
• Causes random and significant delays for both sleeper
and waker though
• There’s a cure: insomnia! And continuous activity.
• Uses a lot of power, but it’s a winter, after all. And
there’s no choice, threads are affine to CPU cores
already.
• System scheduler can not balance the load anymore,
we have to do that manually
• There are some patterns to help minimise the work
involved

25. Threads never sleep

26. Threads never sleep

27. IO
• Specialized NICs with userspace API
• Application has an exclusive ownership over
a NIC
• No interrupts>no kernel>no unpredictability
• Non-blocking async reads from network
• 10Gbit bypass NICs are cheap
• No local IO (bye-bye, filesystem)
• File access causes unpredictable jitter
• Move all IO to the network, incl logging

28. Summary
• There’s no ideal environment to develop latency-
restricted apps
• Java is far from this ideal too
• But the balance of features enables Java’s usage in
this complex uneven area
• ... and help to make some money too
• Unfortunately, there’s no room for Scala (yet) :(
• Thank you! (and we’re hiring)
viktor@algotradinghub.com @vsovietov