Running Java in Containers, tips and tricks.

1. ¡Bienvenidos
!

2. Java in containers
And a little bit about Container Internals
Martín Baez

3. SOBRE MÍ
Nombre: Martín Báez
Nacionalidad: Argentino
Profesión: Technical Leader
Especialidad: Java backend /
Software Architecture
FOTO
ORADOR

4. Limiting resources in
Linux

5. Limiting process resources in Linux - chroot
● chroot: It is an operation that changes the apparent root directory for the current
running process and their children. A program that is run in such a modified
environment cannot access files and commands outside that environmental directory
tree. This modified environment is called a chroot jail.
• It is a way to isolate apps
• Change the process directory root
• chroot /path/to/newRootDir
• Deboostrap
• A simple tool to install a bases debian system in any subdirectory
• https://wiki.debian.org/Debootstrap
Martín Baez
Debooststrap example
# mkdir /stable-chroot
# debootstrap stable /stable-chroot
http://deb.debian.org/debian/

6. Limiting process resources in Linux - cgroups
● cgroups
• Started in 2006 with the name “process containers”(Paul Menage and Rohit Seth)
• Included in Linux Kernel when version 2.6.24 was released
• "Cgroups allow you to allocate resources — such as CPU time, system memory,
network bandwidth, or combinations of these resources — among user-defined groups
of tasks (processes) running on a system"
• All processes on a Linux system are child processes of a common parent:
the init process(or systemd), which is executed by the kernel at boot time and starts
other processes (which may in turn start child processes of their own). Because all
processes descend from a single parent, the Linux process model is a single hierarchy,
or tree.
• Additionally, every Linux process except init inherits the environment (such as the PATH
variable)
• Many different hierarchies of cgroups can exist simultaneously on a system. If the Linux
process model is a single tree of processes, then the cgroup model is one or more
separate, unconnected trees of tasks (i.e. processes).
•
Martín Baez

7. Limiting process resources in Linux – cgroups(cont.)
● cgroup model
• each hierarchy is attached to one or more subsystems.
• A subsystem represents a single resource
● cgroup subsystems
• cpu
• To provide cgroup tasks access to the CPU.
• memory
• Sets limits on memory use by tasks in a cgroup
• devices
• Allows or denies access to devices by tasks in a cgroup.
• ns
• Namespaces subsystem
• Others
• Freezer, net_cls,net_prio,perf_event
Martín Baez

8. Limiting process resources in Linux –
cgroups(Example)
• Let's create two groups:
• Assing 70% cpu time to one cgroup(red)
• Assing 70% cpu time to the other(blue)
We will create two cgroups in cpu subsystem: cpu_high and cpu_low
# mkdir /sys/fs/cgroup/cpu/cpu_high
# mkdir /sys/fs/cgroup/cpu/cpu_low
cgroup cpu_high:70% of CPU time and cpu_low will get 30% cput time
# echo 717 > /sys/fs/cgroup/cpu/cpu_high/cpu.shares
# echo 307 > /sys/fs/cgroup/cpu/cpu_low/cpu.shares
taskset command allow us to attach a process to a core.
# taskset -c 0 xterm -bg blue &
# taskset -c 0 xterm -bg red &
Martín Baez

9. Limiting process resources in Linux –
cgroups(Example)
Martín Baez
$ echo $$ > /sys/fs/cgroup/cpu/cpu_high/cgroup.procs
$ md5sum /dev/urandom &
$ echo $$ > /sys/fs/cgroup/cpu/cpu_low/cgroup.procs
$ md5sum /dev/urandom &

10. Limiting process resources in Linux – Namespaces
● They allow for isolation of global system resources between independent processes.
 For example, the PID namespace isolates the PID number space. This
means that two processes running on the same host can have the same
PID!
● Without namespaces, a process running in container A could, for example, umount an
important filesystem in container B.
● The idea is that you can't interfere with something if it’s not visible to you.
Martín Baez

11. GNU Linux Tools
/proc virtual file system
● The /proc filesystem contains a illusionary filesystem. It does not exist on a disk.
Instead, the kernel creates it in memory. It is used to provide information about the
system.
● $ man proc
● /proc/1
● /proc/cpuinfo
● /proc/meminfo
● /proc/stat
● There are many commands that do little more than read the above files and
format them for easier understanding
● top
● ps
● free
Some System calls collect information from the environment in GNU Linux
Martín Baez

12. Containers

13. Containers – Motivation (from the dev part of
devops)
● You don’t need to install a bunch of language environments on your system. You can
simply run the ruby / python / java application inside docker.
● Consistent development environments for the entire team.
● Different versions of same programming language without having to hack arounds your
machine.
● Think of many jvm version and vendors in you laptop(JAVA_HOME, PATH....)
● If it runs in your container, it will run on your Linux server
● If you’re having a hard time building / compiling the application code, then build it
inside Docker
● https://cloud.google.com/containers/
Martín Baez

14. Containers – They are not a new idea
● Linux Containers(LXC)
● Solaris Zones
● BSD Jails
● Docker
● Based on LXC in the past (actually it has its ows libraries -> libcontainer)
● OpenVZ
● Heroku
● Awesome! :)
Martín Baez

15. Containers – What they are
● Instead of virtualizing the hardware stack as with the virtual machines approach,
containers virtualize at the operating.
● "Containers are a method of operating system virtualization that allow you to run an
application and its dependencies in resource-isolated processes" (Amazon)
● This means that containers are far more lightweight: they share the OS kernel, start
much faster, and use a fraction of the memory compared to booting an entire OS.
● Docker is the most popular, open-source container format.
● Benefits
 Consistent Environment
 Run Anywhere
 Isolation
Martín Baez

16. Containers vs VMs
● Own network space
● Own network interface
● Can install packages
● Can run processes
● Can be packaged into images
They are not VMS at all
Martín Baez

17. Containers – Example
lxc:
# ls -l /usr/share/lxc/templates/
# lxc-create -t /usr/share/lxc/templates/lxc-alpine -n lxc-alpine
# lxc-start -n lxc-alpine
# lxc-attach -n lxc-alpine
# lxc-stop -n lxc-apline
 It is possible to run docker on lxc.
 In the past docker was based on lxcMartín Baez

18. Containers – How they work?
● In general, Containers running on Linux makes use of kernel namespaces to provide the
isolated workspace called the container.
● When you run a container, Docker creates a set of namespaces for that container.
These namespaces provide a layer of isolation.
 PID namespace for process isolation.
 NET namespace for managing network interfaces.
 IPC namespace for managing access to IPC resources.
 MNT namespace for managing filesystem mount points.
 UTS namespace for isolating kernel and version identifiers.
● They also makes use of kernel control groups for resource allocation and isolation. A
cgroup limits an application to a specific set of resources.
Martín Baez

19. Limiting Containers resources(Docker)
● Docker

Memory

-m / --memory

--memory-swap

If --memory and --memory-swap are set to the same value, this prevents containers
from using any swap

CPU

--cpus

--cpus="1.5", the container is guaranteed at most one and a half of the CPUs

Realtime scheduler
https://docs.docker.com/config/containers/resource_constraints
/
Martín Baez

20. GNU Linux Tools in Docker
Important Issues:
● $ docker run -it -m 512m centos bash
● [root@aba9f6744c3f /]# top
● [root@aba9f6744c3f /]# free –m
● [root@aba9f6744c3f /]# lscpu
● /proc/meminfo, /proc/vmstat and friends are not not cgroup-aware
● They will always display memory numbers from the host system
● Processes inside a container can not rely on free, top and others to determine how
much memory they have to work with
● Auto-scaling is usually a function of how much memory is available INSIDE the
container(this information needs to be accessible from inside the container).

21. Java ergonomics
The JVM provides platform-dependent default selections for the garbage collector, heap
size, and runtime compiler.
● Java processes in Linux don’t behave as expected
 Java ergonomics

“Ergonomics is the process by which the Java Virtual Machine (JVM) and garbage collection
tuning, such as behavior-based tuning, improve application performance.”

The JVM provides platform-dependent default selections for the garbage collector, heap size, and
runtime compiler.”.
● Garbage Collector, Heap, and Runtime Compiler Default Selections
 A class of machine referred to as a server-class machine has been defined as a machine with the following:
 2 or more physical processors
 2 or more GB of physical memory
 On server-class machines, the following are selected by default: Throughput garbage
collector,v Initial heap size of 1/64 of physical memory up to 1 GB, maximum heap size of 1/4
of physical memory up to 1 GB, Server runtime compiler
https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/ergonomics.htm
l

22. An Example
● Java + Spring Boot + Embebed Tomcat
● Reference: https://spring.io/guides/gs/spring-boot-Docker/
Dockerfile:
FROM java:8
ADD /target/example*.jar javaopts.jar# Entry in json format
ENTRYPOINT [“java”, “-jar”, “/javaopts.jar”]
How we run it:
We build the image
$ docker build -t spring-boot-javaopts .
We create and run a container
$ docker run spring-boot-javaopts
Martín Baez

23. Java 9 support for Docker CPU and memory limits
Memory Issues:
• To tell the JVM to be aware of Docker memory limits( in the absence of setting a
maximum Java heap via –Xmx)
• There are two JVM command line options required, -XX:
+UnlockExperimentalVMOptions -
• XX:+UseCGroupMemoryLimitForHeap.
• The -XX:+UnlockExperimentalVMOptions is required because in a future release
transparent identification of Docker memory limits is the goal.
• When these two JVM command line options are used, and -Xmx is not specified, the JVM
will look at the Linux cgroup configuration, Docker containers also use
cgroups configuration for CPU limits too.

24. Java 9 support for Docker CPU and memory limits
CPU Issues:
• As of Java SE 8u131, and in JDK 9, the JVM is Docker-aware with respect to Docker CPU
limits transparently.
• If -XX:ParalllelGCThreads, or -XX:CICompilerCount are not specified as command line
options, the JVM will apply the Docker CPU limit as the number of CPUs the JVM sees on
the system. The JVM will then adjust the number of GC threads and JIT compiler threads
just like it would as if it were running on a bare metal system with number of CPUs set as
the Docker CPU limit.
• If -XX:ParallelGCThreads or -XX:CICompilerCount are specified as JVM command line
options, and Docker CPU limit are specified, the JVM will use the -XX:ParallelGCThreads
and -XX:CICompilerCount values.

25. Java 10 support for Docker
Issues:
● https://bugs.openjdk.java.net/browse/JDK-8146115
"When running in a container, the operating system functions used provide information
about the host and do not include the container configuration and limits. The VM and core
libraries will be modified as part of this RFE to first determine if the current running process
is running in a container."

26. References
oDocker Internals: http://docker-saigon.github.io/post/Docker-Internals/
oUnderstanding the Docker Internals:
https://medium.com/@nagarwal/understanding-the-docker-internals-7ccb052
ce9fe
oLimit a container's resources
https://docs.docker.com/config/containers/resource_constraints/
oJava inside docker: What you must know to not FAIL:
https://developers.redhat.com/blog/2017/03/14/java-inside-docker/
oMemory inside Linux containers: https://fabiokung.com/2014/03/13/memory-
inside-linux-containers/

27. ¡Muchas gracias!

30. ROSA
RIOJUEVES 11 DE OCTUBRE -
18:15 hs
Metropolitano Eventos
Salón Contemporáneo (Junín 501)
Abrimos la INSCRIPCIÓN GRATUITA el lunes 17 de
septiembre!
Sigan nuestras redes para estar atentos
EndavaLat
am
@EndavaLat
am