1. Using CloudStack
with
Clustered LVM
Marcus Sorensen
BetterServers, Inc
2. What's LVM?
● Logical Volume Manager
– Built on Linux device mapper modules
– Physical volumes create pools called volume
groups
– Volume groups provide extents from which we
obtain logical volumes
– Provides features like resizing, redundancy, thin
provisioning, snapshots
3. What about the clustered part?
● LVM can utilize a cluster manager such as corosync
to syncronize LVM metadata between nodes
● LVM has a configurable locking mechanism for
metadata changes
– /etc/lvm/lvm.conf
● locking_type
– 1 is local (default)
– 2 is external locking library
– 3 is built-in clustered locking (depends on how LVM was compiled)
● Locking handled by Corosync and DLM
– CMAN in RHEL/CentOS 6.x is corosync
– Ubuntu provides corosync as well
● Read more about configuring a Red Hat cluster
4. Why use CLVM for Primary Storage?
● Provides a way to utilize shared, block-based
storage, such as iSCSI or Fibre Channel
● Allows CloudStack to have a pool from which to
carve out volumes
● Block-based storage is high performance
● Shared storage is required for live migrations
● CLVM is relatively simple; it doesn't require HA
resources or any of the more difficult configurations
of a typical RedHat cluster.
5. Setting up the storage
1. Install a base RHEL/CentOS operating system
on all cluster nodes
2. Install cluster software
1. RHEL/CentOS: “yum install lvm2-cluster”
2. Ubuntu: “apt-get install clvm”
3. Edit /etc/lvm/lvm.conf to change locking_type
4. Edit or create /etc/cluster/cluster.conf
5. Start services
6. Create volume group(s)
7. Add into CloudStack
6. /etc/lvm/lvm.conf
● Set locking_type to 3
● Other tunables to be aware of:
– wait_for_locks
– filter
8. Start Services
● RHEL/CentOS
– service cman start
– service clvmd start
● Ubuntu
– service cman start
– service clvm start
● Check services:
– cman_tool status
– cman_tool services
● Create volume groups
– Just use normal LVM commands on shared block devices
10. Under the hood... briefly
● Volume groups and logical volumes are managed as storage
pools by libvirt in LibvirtStorageAdaptor
● Pools are activated/deactivated, created/destroyed
dynamically as needed
11. Limitations/bugs
● Snapshots consist of LVM snapshot, copy
snapshot to secondary, delete snapshot
● Due to nature of templating, templates
unnecessarily copied to primary storage
before use (fixed in master)
● New dm-thin module won't be of much use
with current implementations for thin-
provisioned LVM volumes
12. Looking to the future...
● Refactor to use Edison's storage plugin
framework
● Utilize dm-thin for
backingstore/clone/snapshot functionality
● Could provide (limited) dm-thin based
snapshot functionality directly via device-
mapper