1. XenServer 6.1
Storage XenMotion
Christian Ferber
Sr Systems Engineer Cloud Infrastructure
November, 2012

2. Feature Overview: Use cases
1. Upgrade a storage array
2. Upgrade a pool with VMs on local storage
3. Provide tiered storage arrays
4. Rebalance VMs between XenServer pools, or CloudStack clusters
“The Cloud” was the major use case we had in mind when designing this.
© 2012 Citrix

3. Live Storage XenMotion
Upgrading VMs from Local to Shared Storage
Live
Virtual
Machine
XenServer Hypervisor
VDI(s)
Local
Storage
FC, iSCSI, NFS SAN XenServer Pool
© 2012 Citrix

4. Live Storage XenMotion
Moving VMs within a Pool with local-only storage
Live
Virtual
Machine
XenServer Hypervisor XenServer Hypervisor
VDI(s)
Local Local
Storage Storage
XenServer Pool
© 2012 Citrix

5. Live Storage XenMotion
Moving or rebalancing VMs between Pools (Local  SAN)
Live
Virtual
Machine
XenServer Hypervisor XenServer Hypervisor
XenServer Hypervisor XenServer Hypervisor
XenServer Hypervisor XenServer Hypervisor
VDI(s)
Local
Storage FC, iSCSI, NFS SAN
XenServer Pool 1 XenServer Pool 2
© 2012 Citrix

6. Live Storage XenMotion
Moving or rebalancing VMs between Pools (Local  Local)
Live
Virtual
Machine
XenServer Hypervisor XenServer Hypervisor
XenServer Hypervisor XenServer Hypervisor
XenServer Hypervisor XenServer Hypervisor
VDI(s)
Local Local
Storage Storage
XenServer Pool 1 XenServer Pool 2
© 2012 Citrix

7. Storage XenMotion Features
• Destination network can be selected
• Transfer network can be selected
• Migration doesn’t interfere with VM powerstate ops
• Extra network IO won’t interfere with Xapi’s management interface
• Up to 3 concurrent Storage Migrations per host
• Up to 6 VDIs on each VM
• Supported on all VHD based SRs (Local EXT, Local LVHD, iSCSI, FC)*
© 2012 Citrix * Not currently supported with StorageLink SRs

8. Feature Overview: GUI walkthrough
© 2012 Citrix

9. Feature Overview: GUI walkthrough
© 2012 Citrix

10. DEMO
Storage XenMotion
© 2012 Citrix

11. Storage XenMotion
Deep dive

12. Feature Overview: CLI walkthrough
• xe vm-migrate
ᵒ New params: remote-address, remote-username, remote-password,
remote-network, vif, vdi
• Extends the original vm-migrate command
• Bold params are required to enable cross-pool migration
• vif and vdi map VIFs to target networks and VDIs to target SRs
• remote-network specifies the network used for data transfer
• Can use host/host-uuid to specify host on pool to send VM
• xe vdi-pool-migrate
ᵒ Params: uuid, sr-uuid
• uuid of target VDI
• sr-uuid of destination SR
© 2012 Citrix

13. Feature Overview: Bird’s-eye view
Cross-pool migration and VDI migration consist of the following:
1. Synchronously mirror VDIs between source and destination
2. Create new VM object on destination pool (new ref, same uuid)
3. When copy complete, migrate VM as usual
Note: VDI migrate implemented with “localhost” cross-pool migrate!
© 2012 Citrix

14. Architecture: Intro to VHDs
• Many SRs implement VDIs as VHD trees
• VHDs are a copy-on-write format for storing virtual disks
• VDIs are the leaves of VHD trees
• Interesting VDI operation: snapshot (implemented as VHD “cloning”)
RW RO
A
RO RW
A: Original VDI B A
B: Snapshot VDI
© 2012 Citrix

15. no color = empty
Storage XenMotion gradient = live
VM
root
SOURCE DESTINATION
© 2012 Citrix

16. Architecture: VDI operations
• For each VDI:
ᵒ Snapshot VDI and synchronously mirror all subsequent writes to destination SR
ᵒ Copy the snapshot to destination SR
ᵒ Finally, compose those writes onto the snapshot on the destination SR
• Continue to mirror all new writes
• Each of these operations occurs sequentially for each VDI, but each VDI
mirror continues until the VM migration is complete
• VM memory is copied only after final VDI compose is complete
VDI 1: snapshot & start mirroring
VDI 1: copy snapshots
VDI 2: snapshot & start mirroring
VDI 2: copy snapshots
Copy VM memory
© 2012 Citrix

17. Architecture: Summary
• New components: xenopsd, SMAPI v2, tapdisk NBD plugin
• Cross-pool migration comprised of VDI snapshotting, copying, and mirroring
• VDI-migration implemented as a “localhost” cross-pool migration
• Both operations require extra, temporary space on source and destination SR
© 2012 Citrix

18. Troubleshooting tips
• Check logs: /var/log/{xensource.log,SMlog,messages}
ᵒ Note: xensource.log and messages both implemented with syslog, so they now have
consistent timestamps!
• xn command: CLI to xenopsd
ᵒ Try ‘xn help’ for documentation.
• tap-ctl command
ᵒ Could be useful for diagnosing problems
© 2012 Citrix

19. Work better. Live better.