Multiple Sites and Disaster Recovery with Ceph Audience: Intermediate Topic: Storage Abstract: Ceph is the leading storage solution for OpenStack. As OpenStack deployments become more mission critical and widely deployed, multiple site requirements are increasing as is the need to ensure disaster recovery and business continuity. Learn about the new capabilities in Ceph that assist customers with meeting these requirements for block and object uses. Speaker Bio: Andrew Hatfield, Red Hat Andrew has over 20 years experience in the IT industry across APAC, specialising in Databases, Directory Systems, Groupware, Virtualisation and Storage for Enterprise and Government organisations. When not helping customers slash costs and increase agility by moving to the software-defined storage future, he’s enjoying the subtle tones of Islay Whisky and shredding pow pow on the world’s best snowboard resorts. OpenStack Australia Day Government - Canberra 2016 https://events.aptira.com/openstack-australia-day-canberra-2016/

1. Andrew Hatfield
Practice Lead - Cloud Storage and Big Data
MULTIPLE SITES AND DISASTER RECOVERY
WITH CEPH
OPENSTACK DAY AUSTRALIA (CANBERRA) NOVEMBER 2016
andrew.hatfield@redhat.com
@andrewhatfield

2. Today’s takeaways
● As OpenStack adoption matures and becomes more mission critical,
Business Continuity increases in importance
● Storage is a key component of Business Continuity
● Ceph can deliver significant benefits already and it's getting even better

3. Please Note
RE: Block / RBD
This is about Business Continuity and Disaster Recovery
This is not yet for High Availability or Fault Tolerance

4. Ceph and OpenStack Overview
Ceph is tightly integrated into
Openstack
Cinder, Glance, Swift, Nova and
Manilla (Tech Preview)
Single storage platform for all
OpenStack needs
Fast booting and cloning

5. Ceph and OpenStack Usage
OpenStack User Survey October 2016 https://www.openstack.org/analytics/

6. Expected capabilities
● Multiple isolated OpenStack environments
● Each site has in-live/in-sync backup of:
○ Glance images
○ Cinder block devices
○ Nova ephemeral disks
● In an event of a failure, any site can recover its data from another site
● Storage architecture based on Ceph

7. Properties:
● Single OpenStack site
● A data recovery site
● Pool names & Cluster FSID match at each site
Challenge:
● Failover procedure
How to recover?
● Promote Secondary, demote Primary and reverse
replication
● Recover data

8. Properties:
● Keystone on the controllers (as usual)
● Individual login on each region/site
● Both sites have each other’s data
● Both sites have the same cluster FSID
Challenge:
● Replicate metadata for images and
volumes
How to recover?
● Promote the secondary site
● Import DB records in the survival site

9. Properties:
● Shared Keystone
● Federated Keystone
● Both sites have each other's data
● Works with 2 sites
● Both sites have with the same cluster
FSID
Challenges:
● Replicate UUID tokens
● MySQL cross-replication over WAN
● Requires low latency and high bandwidth
● Fernet tokens are not ready yet
How to recover?
● Promote the secondary site
● Import DB records in the survival site

10. RBD mirroring
Available with Ceph Jewel and Red Hat Storage 2.0
● New daemon ‘rbd-mirror’ synchronises Ceph images from one cluster to another
● Relies on two new RBD image features:
○ journaling: enables journaling for every transaction on the image
○ mirroring: tells the rbd-mirror daemon to replicate images
● Images have states: primary and non-primary (promote and demote calls)

11. RBD mirroring
Features;
● Can replicate an individual Image or an entire Pool
● Integrates with cinder-replication configuration for OpenStack awareness

12. RBD mirroring write path

13. RBD Mirroring Setup
● Use different cluster names; routable connectivity
● Deploy the rbd-mirror daemon on each cluster
● Same pool name at both sites
● Add peering pool
● Add RBD image settings
○ Enable journaling on image
○ Mirror pool or specific images
Challenges:
● No HA support for RBD-mirror yet
● Two sites only
● LibRBD-only, no current kRBD support

14. What’s Next For Block?
Today is shiny and the future is even brighter!
● Multiple node support
● Multiple site support
● Rbd-mirror proxy support
● Mirror QoS
● Optionally keep deleted images for configurable time
● Configurable replication delay

15. Global Single Object Namespace
Applications
Zone Group
Australia
Zone
South
Zone
North
Applications
RGW
S3 / Swift
RGW
S3 / SwiftAsynchronous two-way replication

16. Try it out!
Ceph Test Drive
http://http://red.ht/2fMaWxr