1. Persistent Storage for Linux Containers
Louis Gordon
Senior Storage Solutions Architect, Red Hat

2. Linux Containers in brief
A Software packaging concept that typically includes an application and all of its runtime dependencies
• Higher Quality Software
Releases
• Shorter Test Cycles
• Easier Application
Management
HOST OS
SERVER
CONTAINER
LIBS
APP
CONTAINER
LIBS
APP
Benefits

3. Greater Portability, Automation, and Integration
Code Registry
Push image
Code & Build Test Deploy
Pull image

4. The Red Hat Stack – From PaaS to Storage
DevOps Tools and User Experience
Language Runtimes and Middleware
Databases and Other Services
Container Orchestration and Management
Container API
Storage
Container Host

5. Typical workloads for Containers
Base: 194 IT operations and development decision-makers at enterprises in APAC, EMEA, and North America
Source: A commissioned study conducted by Forrester Consulting on behalf of Red Hat, January 2015
“For which workloads or application use cases have you used/do you anticipate to use containers?”
Scalable, Cost Effective, Distributed Storage for Containers

6. Why Containers require persistent Storage?
• Web & presentation layers should be stateless ... but ...
• That’s probably < 5% of all application instances in a DC
• Most other infrastructure applications / services require persistence for
storing application and configuration data
• Building a different infrastructure for stateful vs. stateless apps ?
• Modern IT requires Standardization (e.g. Cloud Computing)
• Software-defined DC is a hybrid cloud enabler

7. The requirements for Container Storage?
• Scalable – Scale out capacity & performance
• Resilient – Likely there will be important data
• Flexible – Allow different application access profiles
• Software-defined – To allow flexible deployment models
• Open – For customer choice and lowest TCO
RED HAT
STORAGE➔

8. Current Options for Persistent Storage
Shared Filesystems:
• NFS
• GlusterFS
Block Storage:
• Ceph RBD
• iSCSI or FC
• GCE Persistent Disk
• AWS Elastic Block Store

9. Storage Options Comparison
NFS-Filer GlusterFS Ceph
RBD
iSCSI / FC GCE PD AWS EBS
Scalability - + ++ - ++ ++
Availability + ++ ++ + + +
Cost - ++ + ++ - -
Deploymen
t
Flexibility
- ++ ++ - - -
Data
Privacy
++ ++ ++ ++ -- --
Sharing
Data
++ ++ +/- - - -
Use Cases Generic Data
Sharing
Generic Data
Sharing
Very Large
Deployments,
Object Store
High
Performance
Block
Cheap Block
Storage
Cheap Block
Storage

10. STORAGE APPLIANCE
COMPUTE
NODE + + + +
+
STORAGE
NODE
COMPUTE
NODE + + + +
+ + + +
+
+
+
+
+
+
+
+
Scale-Out Storage
• Increased
Performance &
Capacity
• Avoid Fork-Lift
Upgrades
• Built-In Redundancy

11. Why Bother?
PROPRIETARY
HARDWARE
HARDWARE-BASED
INTELLIGENCE
SCALE-UP
ARCHITECTURE
CLOSED DEVELOPMENT
PROCESS
Common, off-the-shelf hardware
Lower cost, standardized supply chain
Scale-out architecture
Increased operational flexibility
Software-based intelligence
More programmability, agility, and control
Open development process
More flexible, well-integrated technology

12. Persistent Storage requires Redundancy
• Red Hat Storage includes important data redundancy features
• 2-way and 3-way replication
• Erasure Coding
• Geo-Replication
• Snapshots

13. Red Hat Storage and Containers

14. Node 1
NGINX
Container
Node 2
NGINX
Container
Persistent Storage for OSE/RHEL Atomic
Node 1
NGINX
Container
Node 3
MySQL
Container
Fileshare
Fileshare
BlockDevice/
ObjectStore
Atomic Host Atomic HostOPENSHIFTOPENSHIFT
Gluster
Ceph
Gluster

15. Host OS
Container Container
Host OS
Container Container
/share
(Gluster/NFS)
Container Volume mapping options
Gluster Volume
Brick 1 Brick 2 Brick 3 Brick n
/share
(Gluster/NFS)
/share/sub1
➔ /data
/share/sub2
➔ /data

16. Host OS
Container Container
Host OS
Container Container
/shared-volume
(Gluster-Fuse or NFS)
Sharing amongst hosts
/shared-volume /shared-volume /shared-volume /shared-volume
Gluster Volume
Brick 1 Brick 2 Brick 3 Brick n

17. Host OS
Container Container
Host OS
Container Container
Mapping Ceph RBDs to containers
/shared ➔ /data
Ceph
OSD OSD OSD OSD
CephRBD+
FS
CephRBD+FS
/shared
Ceph RBD + FS
/host-share ➔
/data /local-mnt
/shared ➔ /data
/host-share

18. App Container App Container App Container App Container
Containerized Gluster Storage
/shared-volume /shared-volume
Gluster Container
(privileged)
/bricks/brick2
Gluster Container
(privileged)
/bricks/brick1
/shared-volume
(Gluster-Fuse or NFS)
Host OS Host OS
Gluster Volume
/shared-volume /shared-volume

19. Hyperconverged Containers and Storage
• Lower TCO
• Unified
Orchestration
• Ease of Use
• Greater control
ContainerJBoss
NGINX
NGINX
Container
MASTER
Kubernetes
Node 2
ContainerElastic
Spark
MongoDB
Container
Kubernetes
Node 3
RedisPostgreSQL
Container
Kubernetes
Node 1
Kubernetes
Node 4

20. Container Container Container Container
Containerized Ceph Storage
/local-mnt1
Container
RHCS
Container
RHCS
Ceph RBDs (and Object Storage)
Host OS Host OS
Ceph Pool
/local-mnt2 /local-mnt3 /local-mnt4

21. Flexible Deployment Options
x86 x86 x86
Physical
VM VM VM
Virtualized
VM VM VM
Public / Private Cloud
OpenStack,
AWS, Azure, GCE
VMware, RHEV, KVM
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4
Container
JBoss
NGINX
NGINX Container
MASTER
Kubernetes Node 2
Container
Elastic
Spark
MongoDB Container
Kubernetes Node 3
Redis
PostgreSQL Container
Kubernetes Node 1
Kubernetes
Node 4

22. Customer
Case Studies

23. Customer Case Study: CapitalOne
Business Challenge:
• A leading diversified bank with 65 million customers
• Fast growing business and customer base
• Need to be disruptive and different
• Analytics plays a big role in growth strategy
Solution Description:
• Predefined docker images with a wide variety of analytics
tools
• Self-service Portal for developers to pick and instantiate
• Integrated monitoring and metrics
• Automated lifecycle management of containers
• High availability through MESOS
• Shared and consolidated Storage Platform with Gluster
Solution Benefits:
• More agile application development
• Larger choice of technologies
• Optimal resource usage and
performance

24. Customer Case Study: CapitalOne

25. Customer Case Study: Verizon
D
o
c
k
e
r
c
o
n
t
a
i
n
e
r
s
r
u
n
n
i
n
Business Challenge:
• Verizon is the largest provider of cellular services in the United States with
more than 100 million subscribers. Pretty much every subscriber takes
‘selfies’ and snaps of the kittens and these need to be backed up. Whilst many
users use Apple’s iCloud, or Yahoo’s Flickr, Verizon also has their own branded
Cloud offering – Verizon Cloud.
• Verizon wished to use a file-based format rather than converting all these
images to Objects and to have an extremely efficient architecture in terms of
the use of their server infrastructure.
Solution Description:
• Simplified Deployment via containers
• Seamless upgrade and rollback
• Dedicated and Containerized Gluster for FSaaS
• Dynamic scaling (up) of capacity as needed
• Performance and health metrics collection via container monitoring agents
and sending alerts.
Solution Benefits:
• Quicker time to market for new services
• Increased subscriber ‘stickiness’ and improving customer
satisfaction.
• Reduced infrastructure Costs
• Infrastructure can grow as demand grows

26. ETH/IB
app app app
app app app
Converged Computing Architecture
Mixed App and Storage Workloads
App-only servers
Converged servers
app
app +
Storage-only servers
app app app app
app app app app
app app app app
app app
app
Storage stack imposes only
3% - 10% load on compute
processing
• Applications and storage
stacks can co-exist on same
compute substrate.
• This achieves higher server
utilization and lower
operational costs across the
cloud
Customer Case Study: Verizon

27. Summary
• Most containerized applications will require Persistent Storage
• Software-defined Storage allows hyper-convergence for applications and
storage
• Red Hat Storage and OpenShift Enterprise provide a complete PaaS solution
with full deployment flexibility from on-premise to hybrid clouds

28. Thank You