Learn how ScyllaDB Cloud is moving to serverless, transforming its single tenant deployment model into a multi-tenant architecture based on Kubernetes. Discover the engineering innovation required, and the user value of the new architecture, including use of encryption (both at flight and at rest), performance isolation, and the capability to scale elastically.
2. Yaniv Kaul
■ Joined ScyllaDB in October 2022
■ Previously, at Red Hat, managed
■ Storage (Ceph, cloud storage),
■ Virtualization (OpenStack)
■ Performance and Scale.
■ Years ago, worked with ScyllaDB founders on the KVM
hypervisor
5. ScyllaDB Cloud
■ Our managed ScyllaDB clusters offering
■ Provides clusters running on AWS or GCP clouds
■ Uses whole virtual machine(s) per tenant
■ Use our infrastructure or ‘bring-your-own-account’ nodes
■ Lifecycle, monitoring, logging, alerting - overall management by our DevOps and
Support teams
■ Free trial also available!
cloud.scylladb.com
6. ScyllaDB Cloud - K8S based
The next generation ScyllaDB Cloud is moving to run ScyllaDB workloads on Kubernetes (K8S).
■ Infrastructure components (storage, network, security, monitoring, logging and more)
deployment, management are handled by Kubernetes
■ Provides seamless integration with cloud services
■ Based on the open source ScyllaDB K8S operator
17. Challenges With K8S
ScyllaDB on K8S has some of the ‘usual’ challenges and some ‘unusual’ ones:
■ Stateful application (a database)
■ Strict requirements from the operating system
■ Deployment requirements
18. The control plane is mostly running on a separate system
■ ScyllaDB Cloud management system was extended to treat serverless as
yet another cloud provider
■ Monitoring and logging are kept outside the operational K8S cluster running
the workloads
Management
19. First, need to provision a K8S node (worker), that will be used to run tenants’
ScyllaDB nodes:
■ RAID(0) configuration, OS tuning, etc.
■ Disk IO benchmarking, to get the baseline performance numbers.
■ Avoid ‘lemons’
■ Distribute baseline IO performance between tenants (per tenant CPU
shares)
Data Plane / Workers
20. The ScyllaDB K8S operator deploys a complete ScyllaDB cluster:
■ No. of nodes, size (CPU, memory, disk) , distribution, replication factor
(RF=3 for the time being), and other configuration items.
■ Configure / initialize node, wait for cluster formation
■ Manager (agent) & configuration
■ Monitoring
■ Drivers proxy (‘SNI proxy’)
Tenant Deployment
21. ■ Virtual machines forces a fixed cores/memory/disk relationship
■ Scaling can only be performed by adding or removing nodes
■ With K8S, we abstract underlying hardware and provide much more
flexibility.
Breaking the Fixed Compute Storage
Relationship
22. ■ Memory - each tenant (Pod) get their own (request/limit)
■ CPU - each tenant (Pod) get their own (request/limit)
■ Disk - each tenant get their own XFS mount point, with quota assigned
■ Scylla’s IO scheduler knows how much IOPS to give to each tenant
■ KMS is used to encrypt each tenant’s data
Tenant Isolation
26. The different CQL drivers connect to the cluster via a proxy, known as the SNI
(Server Name Indication) proxy:
■ Cluster data is stored in a YAML file, that clients use to connect to the
cluster.
■ Using TLS, the clients use the SNI field to indicate to which ScyllaDB node
they will connect to.
■ They still connect to all shards (shard-aware)
Connection to the Cluster
28. As part of the ‘S301: ScyllaDB Operations’ course you can learn more about our K8S operator
and get some hands-on experience working with it.
ScyllaDB University Content
29. Thank You
Stay in Touch
Yaniv Kaul
yaniv.kaul@scylladb.com
@YanivKaul
github.com/mykaul
www.linkedin.com/in/ykaul/