2. Boring slide
● At work
○ Staff Kubernetes Integration Engineer
○ @Ondat (former StorageOS)
○ Operator, Scheduler, Controller
and Automation
● At IT space
○ Many years of DevOps, cloud
and containerization
○ Several years of Go, Java, so on
■ (why others than BASH exists?)
○ OSS devotee
○ Known as @mhmxs
6. Ondat - Key features
● Built-in compression
● Encryption at rest
● Replication
● Data locality
● Shared volumes
● Snapshots
● Storage pooling
● Topologies
● Web Portal
● Free community edition [unlimited clusters, nodes and 1TB capacity]
● Rolling-upgrades - Tech preview
7. Persistent Volumes
● Implementation details are out of scope for Kubernetes
● Container Storage Interface [CSI]
○ Runs on “every” node, Kubelet calls via Unix socket
○ Plugin capabilities endpoint
○ Provision, Attach, Resize, Snapshot
○ Most runs in privileged mode
○ Dependency out of Devs control
8. Persistent Volumes
● storage.k8s.io.StorageClass/v1
○ Cluster wide resource
○ Defines common (low level) details of Persistent Volumes
■ Provisioner: ID of CSI driver
■ Parameters: passed to CSI driver
■ Reclaim policy: Retain, Delete
■ Allow volume expansion
■ Mount options
■ Volume binding mode: Immediate, WaitForFirstConsumer
○ Usually managed by Ops team
9. Persistent Volumes
● core.PersistentVolumeClaims/v1
○ Namespaced resource
○ Still not the volume itself (template)
○ Defines details of Persistent Volumes
■ Storage class or default
■ Access mode:
● ReadWriteOnce - RW single node
● ReadWriteOncePod -RW single pod
● ReadOnlyMany - R multi node
● ReadWriteMany - RW multi node
10. Persistent Volumes
■ Volume mode: Filesystem, Block
■ Capacity
■ Selector: matching PersistentVolume
● Both direction works: PVC->PV, PV/Snapshot <-PVC
● PVC with a non-empty selector can't have a PV dynamically
provisioned for it.
○ Yes, you are right, PVC maintainer has to know CSI driver details
○ Usually managed by Dev team
11. Persistent Volumes
● core.PersistentVolume/v1
○ Cluster wide resource
○ No direct connection with workload
○ Full with deprecated in-tree plugins
■ Host path
■ Local
■ CSI
○ Where data lives
○ Dynamic provisioning
■ Managed by CSI driver
○ Static provisioning
■ Managed by Ops team
13. Persistent Volumes
● storage.k8s.io.VolumeAttachment/v1alpha1
○ Namespaced resource
○ Binds volume to exact node
○ Usually managed by ??? team - collaboration
■ Attacher - depends on CSI driver
■ Nodename - needs infra knowledge
■ PVC name - known by Devs
15. Difficulties
● Hard to be infrastructure independent
○ Different environments has different storage backends
○ Different storage backends needs different manifests
● Knowledge/permission gaps
○ Developers has to know different storage backends
○ Developers has to raise tickets to create cluster resources
● Manifest maintenance
○ Overlaying or templating, that is the question
● Scaling
○ Needs extra care on wide spectrum:
■ Manual
■ …
■ Full GitOps pipeline [monitoring, trigger, update, push, pull, apply]
24. Downside
● Not production ready at the moment
● Only AWS EBS and Ondat CSI drivers are supported at the moment
○ New driver is a few bytes of “WebAssembly”
○ Bring Your Own Driver-Integration
● Not every application supports numeric order of directories
● StatefulSets support is incomplete
● Not well documented
25. ● Not production ready at the moment
● Only AWS EBS and Ondat CSI drivers are supported at the moment
○ New driver is a few bytes of “WebAssembly”
○ Bring Your Own Driver-Integration
● Not every application supports numeric order of directories
● StatefulSets support is incomplete
● Not well documented
26. Future plans
● Inspire community to implement CSI drivers
● Implement full integration of StatefulSets
● Automatic snapshot feature would be nice
● Try and document edge case scenarios
● Improve performance at high scale
