1. The Future of GlusterFS and Gluster.org
John Mark Walker
GlusterFS Community Guy
Red Hat, Inc.
January 25, 2012
2. The Roots of GlusterFS
● Distributed storage solutions difficult to find
● Decided to write their own
● No filesystem experts – Pro & Con
● Applied lessons from microkernel architecture
– GNU Hurd
January 26, 2012 The Future of Gluster.org - John Mark Walker
3. The Roots of GlusterFS
● All storage solutions were either
● Too expensive. or...
● Not scalable, or…
● Single purpose, or…
● Don’t support legacy apps, or…
● Don't support new apps, or...
● Do some combo of the above, but not very well
4. The Roots of GlusterFS
● The challenge:
● Create a storage system that was…
– Scalable
– Seamlessly integrated in the data center
– Future-proof
● The solution: GlusterFS
● Scalable, with DHT
● POSIX-compliant
● Stackable
● User-space
5. GlusterFS Client Architecture
● Creating a file system in user space
● Utilizes fuse module
– Kernel goes through fuse, which hands off to glusterd
glusterd Applications
Linux kernel
Fuse Ext4 … …
6. No Centralized Metadata
Client A Client B Client C
Server X Server Y Server Z
Extended Attr. Extended Attr. Extended Attr.
Files Files Files
7. What is a Translator?
● Add/remove layers FUSE Interface Layer
● Reorder layers Performance Layer
● Move layers between Distribution Layer
client and server
Replication Layer
● Implement new layers
Protocol Layer
● e.g. encryption
● Replace old layers Local Filesystem Later
● e.g. replication
8. Some Features
● Distributed, replicated and/or striped volumes
● Global namespace
● High availability
● Geo-replication
● Rebalancing
● Remove or replace bricks
● Self healing
● volume profile and top metrics
10. Red Hat Invests in GlusterFS
● Unstructured data volume to grow 44x by 2020
● Cloud and virtualization are driving scale-out
storage growth
● Scale-out storage shipments to exceed 63,000
PB by 2015 (74% CAGR)
● 40% of core cloud spend related to storage
● GlusterFS-based solutions up to 50% less than
other storage systems
11. Red Hat Invests in GlusterFS
● GlusterFS adds to the
JBoss
Red Hat stack
● Complements other
RHEL
Red Hat offerings
● Many integration points
Bare
RHEV Clouds
● More engineers Metal
hacking on GlusterFS
GlusterFS Unified Storage
than ever before
12. Red Hat Invests in GlusterFS
● Acceleration of community investment
● GlusterFS needs to be “bigger than Red Hat”
● Transformation of GlusterFS from product to project
– From “open core” to upstream
● More resources for engineering and community
outreach
● Red Hat's success rests on economies of scale
– Critical mass of users and developers
13. Join a Winning Team
“Join me, and
together, we can
rule the galaxy...”
● We're hiring hackers and engineers
● Looking for community collaborators
● ISVs, students, IT professionals, fans, et
al.
15. The Gluster Community
Global adoption ● 300,000+ downloads
● ~35,000 /month
● >300% increase Y/Y
● 1000+ deployments
● 45 countries
● 2,000+ registered
users
● Mailing lists, Forums, etc.
16. The Gluster Community
● Why are we changing?
● Only 1 non-Red Hat core contributor
– There were 2, but he acquired us
● Want to be the software standard for distributed
storage
● Want to be more inclusive, more community-driven
Goal: create global ecosystem that
supports ISVs, service providers and more
17. Towards “Real” Open Source
● GlusterFS, prior to acquisition
● “Open Core”
● Tied directly to Gluster products
– No differentiation
● Very little outside collaboration
● Contributors had to assign copyright to Gluster
– Discouraged would-be contributors
18. Towards “Real” Open Source
“Open Core”
● All engineering controlled by
project/product sponsor
● No innovation outside of
Commercial Product
core engineering team
● All open source features
also in commercial product
● Many features in
Open Source
Commercial product not in Code
open source code
19. Towards “Real” Open Source
“Real” Open Source
● Many points of collaboration
and innovation in open
source project
Open Source Code
● Engineering team from
multiple sources
● Project and product do not
completely overlap
● Commercial products are Commercial Products
hardened, more secure and
thoroughly tested
20. Towards “Real” Open Source
“Real” Open Source
● Enables more innovation on
the fringes
Fedora Linux
● Engineering team from
multiple sources
● Open source project is
“upstream” from commercial
product
● “Downstream” products are RHEL
hardened, more secure and
thoroughly tested
21. Towards “Real” Open Source
“Real” Open Source
● Enables more innovation on
the fringes
GlusterFS
● Engineering team from
multiple sources
● Open source project is
“upstream” from commercial
product
● “Downstream” products are Red Hat Storage
hardened, more secure and
thoroughly tested
23. What's New in GlusterFS 3.3
● New features
● Unified File & Object access
● Hadoop / HDFS compatibility
● New Volume Type
● Replicated + striped (+ distributed) volumes
● Enhancements to Distributed volumes (DHT translator)
● Rebalance can migrate open files
● Remove-brick can migrate data to remaining bricks
● Enhancements to Replicated volumes (AFR translator)
● Change replica count on an active volume, add replication to distribute-only volumes
● Granular locking – Much faster self-healing for large files
● Proactive self-heal process starts without FS stat
● Round-trip reduction for lower latency
● Quorum enforcement - avoid split brain scenarios
GlusterFS 3.3 ETA in Q2/Q3 2012
24. File and Object Storage
● Traditional SAN / NAS support either
file or block storage
● New storage methodologies
implement RESTful APIs over HTTP
● Demand for unifying the storage
infrastructure increasing
● Treats files as objects and volumes
as buckets
● Available now in 3.3 betas
● Soon to be backported to 3.2.x
● Contributing to OpenStack project
● Re-factored Swift API
25. Technology Integrations
GlusterFS used as VM storage system
Mobile Apps. Web Clients. Enterprise Software Ecosystem
● Pause and re-start VM’s, even on another API Layer
hypervisor
…
● HA and DR for VM’s Compute
● Faster VM deployment
● V-motion –like capability Unified File &
Object Storage
Shared storage ISOs and appliances
● oVirt / RHEV
● CloudStack
● OpenStack OpenStack Imaging Services
Goal: The standard for cloud storage
26. HDFS/Hadoop Compatibility
● HDFS compatibility library
● Simultaneous file and object access within Hadoop
● Benefits
● Legacy app access to MapReduce applications
● Enables data storage consolidation
● Simplify and unify storage deployments
● Provide users with file level access to data
● Enable legacy applications to access data via NFS
● Analytic apps can access data without modification
27. The Gluster Community
● What is changing?
● HekaFS / CloudFS being folded into Gluster project
– HekaFS == GlusterFS + multi-tenancy and SSL for auth
and data encryption
– HekaFS.org
– ETA ~9 months
29. GlusterFS Advisory Board
● Advisory board
● Consists of industry and community leaders from Facebook, Citrix,
Fedora, and OpenStack
– Richard Wareing, Storage Engineer, Facebook
– Jeff Darcy, Filesystem Engineer, Red Hat; Founder, HekaFS Project
– AB Periasamy, Co-Founder, GlusterFS project
– Ewan Mellor, Xen Engineer, Citrix; Member, OpenStack project
– David Nalley, CloudStack Community Mgr; Fedora Advisory Board
– Louis Zuckerman, Sr. System Administrator, Picture Marketing
– Joe Julian, Sr. System Administrator, Ed Wyse Beauty Products
– Greg DeKoenigsberg, Community VP, Eucalyptus; co-founder, Fedora
– John Mark Walker, Gluster.org Community Guy (Chair)
30. Gluster.org Web Site
● Services for users and developers
● Developer section with comprehensive docs
● Collaborative project hosting
● Continuing development of end user documentation
and interactive tools
● Published roadmaps
● Transparent feature development
32. Gluster.org Services
● Gluster.org
● Portal into all things GlusterFS
● Community.gluster.org
● Self-support site; Q&A; HOWTOs; tutorials
● Patch review, CI
● review.gluster.com
● #gluster
● IRC channel on Freenode
33. Development Process
● Source code
● Hosted at github.com/gluster
● Bugs and Feature Requests
● Bugzilla.redhat.com – select GlusterFS from menu
● Patches
● Submit via Gerritt at review.gluster.com
● See Development Work Flow doc:
● gluster.org/community/documentation/index.php/Development_Work_Flow
34. Thank You
● GlusterFS contacts
● Gluster.org/interact/mailinglists
● @RedHatStorage & @GlusterOrg
● #gluster on Freenode
● My contact info
● johnmark@redhat.com
●
Twitter & identi.ca: @johnmark