Graham Mainwaring and Robert Hodges summarize management of ClickHouse on Kubernetes using the ClickHouse Kubernetes Operator and introduce a new UI for it. Presented at the 15 Dec '22 SF Bay Area ClickHouse Meetup.

1. Data Warehouses in Kubernetes Visualized
Introducing the ClickHouse
Kubernetes Operator UI
Graham Mainwaring
Robert Hodges
1

2. © 2021 Altinity, Inc.
Presenter Bios and Altinity Introduction
ClickHouse support and services including Altinity.Cloud
Authors of ClickHouse Operator for Kubernetes
and other fine open source projects
Robert Hodges
Altinity CEO
30+ years on DBMS plus
virtualization and security.
ClickHouse is DBMS #20
Graham Mainwaring
Senior Cloud Eng
25 years of enterprise
development; author of
Ansible Receptor project

3. © 2021 Altinity, Inc.
What is
Kubernetes?
3

4. © 2021 Altinity, Inc.
“Kubernetes is the new Linux”
Actually it’s an open-source platform that is:
● An orchestrator for container-based apps
● Infrastructure as code
● A great cluster manager
● A terminal endpoint for CI/CD automation

5. © 2021 Altinity, Inc.
Where Kubernetes runs…
5
Dev Environment
(Laptop/VM)
Self-managed
Managed

6. © 2021 Altinity, Inc.
How Kubernetes manages applications
Kubernetes
Resources
Server
Storage
Server
Storage
Server
Storage
. . .
Physical
Nodes
kube-system default my-namespace . . .

7. © 2021 Altinity, Inc.
A simple distributed application
Load
Balancer
Process
#1
Process
#3
Process
#2
Storage
Storage
Storage
Traffic

8. © 2021 Altinity, Inc.
Defined using Kubernetes resources
Pod
“svc-1”
Persistent
Volume
Service
“svc”
Stateful
Set
Persistent
Volume
Claim
Persistent
Volume
Persistent
Volume
Pod
“svc-2”
Pod
“svc-2”
Persistent
Volume
Claim
Persistent
Volume
Claim
Config
Maps
Config
Maps

9. © 2021 Altinity, Inc.
Problem: ClickHouse on Kubernetes is complex!
Zookeeper
Services
Zookeeper-0
Zookeeper-2
Zookeeper-1
Shard 1 Replica 1
Replica
Service
Load
Balancer
Service
Shard 1 Replica 2
Shard 2 Replica 1
Shard 2 Replica 2
Replica
Service
Replica
Service
Replica
Service
User Config Map Common Config Map
Stateful
Set
Pod
Persistent
Volume
Claim
Persistent
Volume
Per-replica Config Map

10. © 2021 Altinity, Inc.
Solution: Kubernetes Operator
kube-system namespace
ClickHouse
Operator
your-favorite namespace
Apache 2.0 source,
distributed as Docker
image
kubectl -f apply
my-cluster.yaml
Best practice deployment
ClickHouse
Resource
Definition
“Adjust reality”

11. © 2021 Altinity, Inc.
Soup-to-Nuts
Cluster Creation
On Kubernetes
11

12. © 2021 Altinity, Inc.
Installing ClickHouse operator
Get operator custom resource definition:
wget
https://raw.githubusercontent.com/Altinity/clickhouse-operator/m
aster/deploy/operator/clickhouse-operator-install-bundle.yaml
Install the operator:
kubectl apply -f clickhouse-operator-install-bundle.yaml
Install in kube-system namespace
Don’t delete -f when there are live clusters!

13. © 2021 Altinity, Inc.
Installing Zookeeper
Get Zookeeper stateful set definition:
wget
https://raw.githubusercontent.com/Altinity/clickhouse-operato
r/master/deploy/zookeeper/quick-start-persistent-volume/zooke
eper-1-node.yaml
Install Zookeeper.
kubectl create ns zoo1ns
kubectl apply -f zookeeper-1-node.yaml -n zoo1ns
Dev/test only!

14. © 2021 Altinity, Inc.
Time to bring up a data warehouse
14
kubectl apply -f first.yaml

15. © 2021 Altinity, Inc.
First.yaml, part 1: Basics
apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
name: "first"
spec:
configuration:
clusters:
- name: "first"
layout:
shardsCount: 2
replicasCount: 2
templates:
podTemplate: clickhouse-stable
volumeClaimTemplate: pd-ssd
zookeeper:
nodes:
- host: zookeeper.zoo1ns
port: 2181
15
Resource type
Shards and replicas
Where is Zookeeper?
Ensure storage is
persistent

16. © 2021 Altinity, Inc.
First.yaml, part 2: Users
users:
# User = demo, Password = topsecret
demo/password_sha256_hex:
53336a676c64c1396553b2b7c92f38126768827c93b64d9142069c10eda7a721
demo/profile: default
demo/quota: default
demo/networks/ip:
- 0.0.0.0/0
- ::/0
16
Password
Profile and quota
IP filters for user

17. © 2021 Altinity, Inc.
First.yaml, part 3: Pods
templates:
podTemplates:
# What is the definition of my server?
- name: clickhouse-stable
spec:
containers:
- name: clickhouse
image: altinity/clickhouse-server:21.8.10.1.altinitystable
# Keep servers on separate nodes!
podDistribution:
- scope: ClickHouseInstallation
type: ClickHouseAntiAffinity
17
Altinity stable
release
Pods on separate nodes

18. © 2021 Altinity, Inc.
First.yaml, part 4: Storage
volumeClaimTemplates:
# How much storage and which type on each node?
- name: pd-ssd
# Do not delete PVC if installation is dropped.
reclaimPolicy: Retain
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
# GKE storage class to allocate pd-ssd storage.
storageClassName: premium-rwo
18
Don’t delete storage!
Storage size
Storage type

19. © 2021 Altinity, Inc.
Accessing our creations
Safety first...Check storage!
kubectl get pvc
And make sure pods are on separate nodes
kubectl get pod -o=custom-columns=NAME:.metadata.name,
STATUS:.status.phase,NODE:.spec.nodeName
Access ClickHouse via load-balancer service
kubectl exec -it chi-first-first-0-0-0 -- bash
clickhouse-client --host=clickhouse-first
--user=demo --password=topsecret

20. © 2021 Altinity, Inc.
After seeing all that…
20
Wouldn’t it be
better if the
operator had a UI?

21. © 2021 Altinity, Inc.
Introducing
Altinity
Dashboard
21

22. © 2021 Altinity, Inc.
Dashboard Goals
● Be able to do the things we just talked about, but without having to remember
all the command syntax.
● Given a Kubernetes cluster, easily deploy the operator and a ClickHouse
Instance in a few mouse clicks.
● Have immediate visibility to the overall state of the deployment.
● Show a focused view of only the items that matter for ClickHouse.
22

23. © 2021 Altinity, Inc.
Demo Time
23
Altinity Dashboard
in action

24. © 2021 Altinity, Inc.
Caveats
● It’s alpha software!
● We are still working on testing and documentation!
● We will be adding more features!
Try it out and tell us what you think by logging issues or on AltinityDB slack
24

25. © 2021 Altinity, Inc.
Learning more
25

26. © 2021 Altinity, Inc.
● Altinity Dashboard
● ClickHouse Kubernetes Operator
● Altinity Documentation
○ Kubernetes operator & ClickHouse blog
● ClickHouse
○ Everything Clickhouse
● Altinity Blog
○ Articles about ClickHouse usage including Kubernetes
More information and references
26

27. © 2021 Altinity, Inc.
Thank you!
We’re hiring
ClickHouse Operator:
https://github.com/Altinity/click
house-operator
Altinity Website:
https://www.altinity.com
Altinity Dashboard:
https://github.com/Altinity/altini
ty-dashboard
27