Kurma is a container runtime that is based on the container instrumentation built into the Apcera Platform. Kurma, and its accompanied “KurmaOS” is our vision of a lightweight, fully containerized operating system. This presentation will cover Apcera’s journey in its container instrumentation. Beginning with the pre-Docker landscape, how it grew over the course of 3+ years, and the “next-gen” adaption of it, where the base container instrumentation has been adapted to stand on its own, and growing it to be used beyond just Apcera’s own usage. Kurma incorporates a lot of lessons learned with both development and operations of a container platform, including building modular vs monolith, extensibility being built in vs built on, and managing a cluster of hosts and containers. We’ll also cover our experiences with introducing it to Kubernetes as another first class runtime provider. Taking how Kurma works and have it work with Kubernetes, and how we’d like to see Kubernetes grow in some of the areas we see Kurma growing.

2. 2016

3. In the beginning…
(2012)

4. The Go Landscape
2012

5. Apcera Platform
The Instance Manager

6. Instance Manager
State Machine Apocalypse
● Started out simple, but naive about the future
● Few small libraries…
● … but all integration logic was central
● 8 states
● 53 function handlers

7. Existing Weight
● Ubuntu base OS
● CAPS deployment
● .deb packaging
● Operational tooling

8. System Skew is a Problem
● Deploys non-atomic
● Different lifecycle per host
● Operational access

10. A New Model
Kurma
● Minimize host dependencies
● Everything is a container
● Simple notions that could be easily extended
● Simple, well defined APIs

11. What is Kurma made of?
Existing
● Go + C
● App Container (AppC)
● Apcera’s existing
instrumentation
Coming soon
● Go + C
● AppC
● libcontainer based
● CNI for networking

12. What is Kurma made of?
Existing
● Go + C
● App Container (AppC)
● Apcera’s existing
instrumentation
Coming soon
● Go + C
● AppC
● libcontainer based
● CNI for networking

13. Delivery
kurmad
● Existing host
● Download and run
● Immediately benefit
● Depends on host kernel
and libc
kurmaOS
● Minimalist distro
● Services as containers
● A/B partition model
● Console is just a
container

14. Delivery
kurmad
● Existing host
● Download and run
● Immediately benefit
● Depends on host kernel
and libc
kurmaOS
● Minimalist distro
● Services as containers
● A/B partition model
● Console is just a
container

15. Kurma
Process Model

16. Kurma
Stager Process

17. Kurma
User Processes

18. Stager
Pluggable Process Orchestration
● Responsible for instrumenting the pod
● Packaged as a signed, trusted ACI image
● Gets own mount and network namespace

19. Stager API
● Simplest unit of work: an executable
● Setup via image ‘Exec’ setting
● Other calls through expected executables
– /opt/stager/run
– /opt/stager/status
– /opt/stager/logs
– ...

20. Kurma
Reusable Unit

21. Kurma
Reusable Unit for Extensibility

22. Networking API
● ACI image
● Passes along JSON configuration
● Executes commands to setup networking on
other containers
– /opt/network/add <ns> <container-id>
– /opt/network/del <ns> <container-id>

23. Kurma
Reusable Unit for Extensibility
/opt/stager/run cni /opt/network/add ...

24. Kurma
Extensibility Through Reuse

25. Kurma
Extensibility Through Reuse

26. Kurma
Extending Boundries with Semantics

27. Kurma
Remote API

28. Kubernetes + Kurma

29. Why?
● Kurma usage outside Apcera
● Increased platform flexibility
● Integrating with broader community

30. Kubelet
● Has existing Runtime interface
● Rich interface for engine communication
● Kubelet is a bit of a leaky abstraction
● Workarounds for Dockerisms

31. Testing
● Mystical
● Documentation gaps
● Excellent Github/PR integration

32. Codebase
● Godep pains
● “hack” directory?
● Documentation gaps
● Interface movement
Runtime.ConvertPodStatusToAPIPodStatus()

33. Kurmanetes
● Maturing Kurma based on Kubernetes needs
– Pods
– Networking
– Image management
● Runtime abstraction nearly complete

34. Kurmanetes
● Done
– Pod management
– Image retrieval and management
● Remaining
– Landing Kurma’s pod/stager branch
– cAdvisor integration
– Integration testing
– Work towards improving the abstraction leaks

35. Questions?

36. Resources
Kurma
kurma.io
github.com/apcera/kurma
Me
ken@apcera.com
@krobertson
We’re hiring for the Kurma team.