With the rapid adoption of microservices, there is a growing need for solutions to manage deployment, resources and data for fleets of microservices. Kubernetes is a resource management framework for containers that is rapidly growing in popularity. Apache Kafka is a streaming platform that makes data accessible to the edges of an organization. It's no wonder the question of running Kafka on Kubernetes keeps coming up!

1. 1
Automate Apache Kafka deployment and
Management with Kubernetes
Damien Gasparina
Solution Architect
Confluent
PARIS - 11 OCTOBRE 2018

2. Who AM I?
●
Solution Architect at Confluent
●
Developer, Engineer, Architect and open source enthusiastic
●
Have been assisting customers worldwide deploying and maintaining the Confluent Platform
I ❤ Building stuff!

3. Who AM I?
●
Solution Architect at Confluent
●
Developer, Engineer, Architect and open source enthusiastic
●
Have been assisting customers worldwide deploying and maintaining the Confluent Platform
😵 typos! 😵

4. Automation

5. InstallationInstallation
Reconfiguration
Upgrade
Scale out
Upgrade
...

6. Manual installation and configuration
Write scripts to maintain configuration
Automated
provisioning &
management
Leverage automated
solutions
3. Tooling
Automation
1. Manual
2. Scripting
4. Automated

7. Time required
The less automated, the
longer it takes…
Scripted
●
Order hardware
●
Create TLS certs
●
Configure script
More…
Requires
coordination
between a few
teams
As a service
Click deploy
No coordination
required
Manual
●
Order hardware
●
Configure OS
●
Configure VM
●
Configure TLS
●
Enable Auth
More…
Requires
coordination
between many
teams
And the
riskier it is

8. I ❤ Automation

10. I ❤ Kubernetes

11. Kubernetes

12. K8s

13. Kubernetes
Works with a range of container tools,
specially Docker
Originally designed by Google and is now maintained
by the Cloud Native Computing Foundation
An open-source system for automating deployment,
scaling, and management of containerized
applications.

15. Kubernetes
Manage all part of the deployment
Network Location StateStorage

20. Helm Chart Helm is a tool that streamlines installing and
managing Kubernetes applications. is basically a
package manager for Kubernetes
And of course, there is a Confluent Helm Chart
packages for the confluent platform:
https://github.com/confluentinc/cp-helm-charts/

26. https://asciinema.org/a/YSEOXRXtnctaH7hfKcudCZJJH
https://tinyurl.com/k8s-cse

27. What about managing Kafka?

28. Kubernetes Operator!

29. Operator
It allows, for example, to use automated
data balancing and automate rolling updates
A method for packaging, deploying and managing a
Kubernetes application.
And there is a Confluent Operator currently in beta,
should be available soon:
https://www.confluent.io/confluent-operator/
Headings

30. $> helm install confluent-operator --set operator.enabled=true
$> helm install -f ./providers/xxx.yaml --set zookeeper.enabled=true
$> helm install -f ./providers/xxx.yaml --set kafka.enabled=true
$> helm upgrade -f ./providers/xxx.yaml --set kafka.enabled=true
Deploy with K8s operator
Upgrade with K8s operator
Delete with K8s
$> helm delete –purge kafka

32. Some recurrent questions...

33. Is it production ready?

34. Ok, I can use it, but
Should I use it?

35. Questions?

36. 36