From RubyConf 2019. How to run parallel ruby with significantly reduced memory usage

1. Parallel Ruby: Managing the
Memory Monster
Kevin Miller kevin.miller@flexport.com

3. We run a lot of background jobs

4. The Great Migration
Old
● 240 Ruby Processes
● 20 Servers
● 12 Processes per Server
● 1 Thread per Process

5. The Great Migration
Old
● 240 Ruby Processes
● 20 Servers
● 12 Processes per Server
● 1 Thread per Process
New
● 400 Ruby Threads
● 10 Servers
● 4 Processes per Server
● 10 Threads per Processes

6. Things started getting… slow

7. Side Lesson
If you deploy every 10 minutes, you can get away with a lot

8. And now back to business

11. Ruby Heap: 40 byte slots

12. Ruby Heap: 40 byte slots
“String”

13. Ruby Heap: 40 byte slots
“String”
Class.new

14. Ruby Heap: 40 byte slots
“String”
Class.new
{a: 2}

15. Ruby Heap: 40 byte slots
“String”
Class.new
{a: 2}
[1]

16. Ruby Heap: 40 byte slots
“String” 12132425
Class.new
{a: 2}
[1]

17. Ruby Heap: 40 byte slots
“String” 12132425
Class.new 121325.224
{a: 2}
[1]

18. Ruby Heap: 40 byte slots
“String” 12132425
Class.new 121325.224
{a: 2} /regular?/
[1]

19. Ruby Heap: 40 byte slots
“String” 12132425
Class.new 121325.224
{a: 2} /regular?/
[1] ...

20. “String” 12132425
Class.new 121325.224
{a: 2} /regular?/
[1] ...

21. “String” 12132425 [“aaaaaaaaa...
Class.new 121325.224
{a: 2} /regular?/
[1] ...

22. “String” 12132425 [“aaaaaaaaa...
Class.new 121325.224
{a: 2} /regular?/
[1] ...

23. C (OS) Heap Pages: 4KB of space per page
Big ArrayBig String

24. “String” 12132425 ArraySlot
Class.new 121325.224
{a: 2} /regular?/
[1] ...
Big
Array
Big
String
[String, String...]

25. “String” 12132425 ArraySlot “aaaa…”
Class.new 121325.224 “aaaa…” “aaaa…”
{a: 2} /regular?/ “aaaa…” “aaaa…”
[1] ... “aaaa…” “aaaa…”
Big
Array
Big
String
[String, String...]

26. “aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“String” 12132425 ArraySlot “aaaa…”
Class.new 121325.224 “aaaa…” “aaaa…”
{a: 2} /regular?/ “aaaa…” “aaaa…”
[1] ... “aaaa…” “aaaa…”
Big
Array
Big
String
[String, String...]

27. “aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“String” 12132425 ArraySlot StringSlot
Class.new 121325.224 StringSlot StringSlot
{a: 2} /regular?/ StringSlot StringSlot
[1] ... StringSlot StringSlot
Big
Array
Big
String
[String, String...]

28. Big
Array
Big
String
[String, String...]
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“String” 12132425 ArraySlot StringSlot
Class.new 121325.224 StringSlot StringSlot
{a: 2} /regular?/ StringSlot StringSlot
[1] ... StringSlot StringSlot
Big
Array
Big
String
[String, String...]

29. Total Slots = 1 for Array + 1024 for the 1024 strings = 1025
Total Ruby Heap Size = 1025 Slots * 40 bytes per slot = ~40KB = ~ .04 MB
Total OS Heap Size =
4 bytes per array elem * 1024 array elems + 1024 bytes per string * 1024 strings =
= 1028KB = ~1MB

31. We expect memory to be around here

32. Big
Array
Big
String
[String, String...]
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“String” 12132425 ArraySlot StringSlot
Class.new 121325.224 StringSlot StringSlot
{a: 2} /regular?/ StringSlot StringSlot
[1] ... StringSlot StringSlot
Big
Array
Big
String
[String, String...]

33. Big
Array
Big
String
[String, String...]
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“aaaa…” “aaaa…” “aaaa…” “aaaa…”
“String” 12132425 ArraySlot StringSlot
Class.new 121325.224 StringSlot StringSlot
{a: 2} /regular?/ StringSlot StringSlot
[1] ... StringSlot StringSlot
Big Array
Big
String Wasted
Space

34. Ruby Threads Are Not Memory Friendly

35. Thread 1
Fragmented
OS Heap 1

36. Thread 1
Fragmented
OS Heap 1
Thread 2

37. Thread 1
Fragmented
OS Heap 1
Thread 2

38. C Default: One OS Heap per Thread
Thread 1 Thread 2
Fragmented
OS Heap 1
Fragmented
OS Heap 2

39. C Default: One OS Heap per Thread
Thread 1 Thread 2 Thread 20
...
Fragmented
OS Heap 1
Fragmented
OS Heap 2
Fragmented
OS Heap 20

40. Option 1: Reduce Number of OS Heaps
Number of OS Heaps (a.k.a Arenas) can be configured with
MALLOC_ARENA_MAX

41. We expect memory to be around here

42. Option 2: Use a Different Memory Allocator
Can switch to jemalloc* over standard glibc malloc with
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
* After installing it via homebrew/apt-get/yum/etc

43. We expect memory to be around here

44. So… What should you do?
Default Reduced Arena Count jemalloc
Pros Out of the Box Works with all ruby
Significant memory
reduction
Best Memory Usage
Cons Heavy Fragmentation Increases contention in
ruby implementations
with true threading
(JRuby)
Requires reinstalling
ruby and installing
jemalloc
Use Cases Single-threaded Ruby
Programs on OS’s you
don’t control
Can set environment
variables but cannot
install packages
Anytime you’re able to

45. So… What should you do?
1) If you control the environment, jemalloc is king
2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2

46. So… What should you do?
1) If you control the environment, jemalloc is king
2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2
“I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is
terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a
miracle occurs: their memory usage drops massively”
Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of
standard Ruby

47. So… What should you do?
1) If you control the environment, jemalloc is king
2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2
“I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is
terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a
miracle occurs: their memory usage drops massively”
“On Ubuntu Linux, with Rails Ruby Bench, jemalloc gives an overall
speedup (not just memory, end-to-end speedup) of around 11%”
Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of
standard Ruby

48. So… What should you do?
1) If you control the environment, jemalloc is king
2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2
Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of
standard Ruby
“I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is
terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a
miracle occurs: their memory usage drops massively”
“On Ubuntu Linux, with Rails Ruby Bench, jemalloc gives an overall
speedup (not just memory, end-to-end speedup) of around 11%”
“I'm another Ruby user that used to have memory bloat problems and
switched to jemalloc as well”

49. Want to Learn More?
Go to Aaron Patterson’s Talk:
“Compacting Heaps in Ruby 2.7”