This document discusses layered orchestration of Redis Enterprise using Kubernetes and Pivotal Cloud Foundry. It describes how Redis Enterprise provides a highly available, scalable database solution that can run on orchestration platforms. Layered orchestration involves Redis Enterprise managing resources internally within containers, while the external orchestrator handles tasks like pod scheduling and scaling. This allows Redis Enterprise to maintain high performance while supporting different deployment environments. The document also outlines Redis Labs' journey to using Kubernetes operators to deploy Redis Enterprise, providing full control over the lifecycle in a Kubernetes-compatible way.

1. Layered Orchestration & Redis Enterprise
for fun and profit
Uri Shachar, Redis Labs

2. ● Redis Enterprise
● Orchestrators: OS, K8s, PCF oh my!
● Layered Orchestration: Putting them together

3. Redis Enterprise: Architecture
3
Distributed Proxies, Single or Multiple Endpoints
Unique multi-tenant container - like architecture enables running hundreds of databases over a single, average cloud
instance without performance degradation and with maximum security provisions.

4. • Faster time to market with continuity between
dev/test and production environments that use
Redis Enterprise
• Highly available, easier to scale, simpler to manage
Redis technology, integrated with orchestration
tools such as PCF, Kubernetes, Mesosphere...
• Node in a container approach — All Redis Enterprise
services inside each container.
Run Redis Enterprise clusters on single or multiple nodes
Redis Enterprise in Containers

5. Node in a Pod Approach
Node 1
Vs
Node 2 Node 3 Node 1 Node 2 Node 3
One pod, multiple services per nodeMultiple pods, multiple services per node
• Resources efficiency - low
• Performance – medium
• Isolation - high
• Resources efficiency - high
• Performance – high
• Isolation - high

6. ● Redis Enterprise
● Orchestrators: OS,K8s,PCF oh my!
● Layered Orchestration: Putting them together

7. Why you need Orchestration!
• Containerized workloads demand choreography
• Microservices adoption increases operational overhead
– with dozens of discrete components
• Orchestration Platforms
– Pivotal Cloud Foundry - Build and Run containerized cloud-native
applications
– Kubernetes – Run containerized cloud-native applications

8. Pivotal Cloud Foundry
• Application PaaS
• Manages storage, compute and networking of containerized
applications and VMs
• You provide
• Application code
• Platform provides
• Application runtime, containerization & scheduling
• Networking & Routing
• Cross cloud / on prem compatibility

9. • Orchestration of containers – Containers as a Service (CaaS)
• Manages storage, compute and networking of containerized applications
• Deployment Life Cycle management
– provides basic health checks for containers
– Allows things like rolling updates
• Has a very declarative definition
– Configure the desired state and it will attempt to get there
Kubernetes Platforms
9

10. ● Redis Enterprise
● Orchestrators: OS, K8s, PCF oh my!
● Layered Orchestration: Putting
them together

11. • Provide a solid primary db solution for end-users
• Stateful application
– Some changes cannot be performed
– Some changes need to mutate the state before applying the actual change
– Data-loss is unacceptable
• Support multiple orchestrated deployment types
– Cloud: GKE, AKS, etc
– Pivotal Cloud Foundry
– Openshift
– PKS
– Vanilla K8s
• Ingress & Persistent storage
• Packaging
Redis Labs Challenges on Orchestrated environments

12. What’s Layered Orchestration ?
Kubernetes PKS
External
Redis Cluster
Orchestration
Node 1
Redis Shards
Node 2 Node N
Internal

13. Why like this?
• Resource management - Orchestration platforms are
designed to be generic.
• Again - Performance is king.
• Last but not least, it allows us to maintain a common
architecture - regardless of running environment, be it bare
metal, VM, K8s, Pivotal Cloud Foundry.
(.… Surprisingly enough, not everybody in the world uses containers…)

14. Layered Orchestrators: Who Does What
• Pod Scheduling & Scaling
• Pod auto-healing
• Failover & shard level scaling
• Configuration & monitoring
• Service discovery
• Upgrade
+
14

15. Started out with 9 static yaml files
– Hard to deploy
– Hard to maintain
– Hard to distribute
– No control over the deployment life-cycle
Our journey to Kubernetes
.yaml
.yaml
Helm - A package manager
– Customized deployment
– Easier to maintain (templating)
– Not fully supported everywhere
– No control over the deployment life-cycle
• Operator
– Simple deployment (2 yaml files)
– Full control over life-cycle
– K8s compatible - no plugin needed

16. Redis
Enterprise
Cluster
Resource
Operator based deployment
Nodes
Persistence
High Availability
DBs and UI endpoints
Secrets
pod-1 pod-2
Pod Disruption Budget
Anti Affinity
stateful set
pod-0
PVPV PV
k8s Node-A k8s Node-B k8s Node-C
Secrets resource
db-1.svc
deployment
Services Rigger
db-2.svc
Management Layer
Operator

17. Thank you!