This talk wants to introduce you to Synnefo, an open source, scalable and production ready cloud stack. It consists of a vm management layer which is google ganeti and is essentially a cluster manager and delivers an integrated management of compute, network and storage. It also runs on of the shelf hardware and delivers live migration without or with shared storage. Synnefo is the orchestration and presentation layer which talks to ganeti via an API. To the users it presents a simple web UI and also exposes an openstack compatible API for automation called cyclades. Also it is possible to do flexible, L2 isolated networking. Also it provides a “dropbox” like storage services called pithos. It features a sync client for different platforms and a web UI. Synnefo offers all the benefits of an amazon like cloud but is geared towards persistent virtual machines. Also it is perfect as a replacement of the traditional virtualization stack and is easy to setup and use.

1. Introduction to Synnefo
open source cloud stack
Thomas Schend, Systems Architect, CompuGroup Medical AG
Open Source Data Center Conference 2014
08. – 10. April, Berlin
4/24/2014 OSDC 20141

2. Agenda
4/24/2014 OSDC 20142
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

3. successfully worldwide in 35 countries for approx. 385,000 customers
24.04.2014 OSDC 20143
 = 19 countries with own offices
 = additional 16 countries with customers
2012 Guidance 2013
Customers 385.000
Group sales € 451 Mio. € 458 – 463 Mio.
EBITDA € 105 Mio. € 97 – 100 Mio.
employees 3.700
Indicators
Best provider of IT solutions for higher quality
and efficiency in health care
Worldwide leader for MIT (Medical Information
Technology)
High profitability and an average of 30% growth
p.a.
Objectives

4. Bio
4/24/2014 OSDC 20144
Systems Architect, responsible for Infrastructure Design
OSS Enthusiast
LPIC-2,MCITP,VCP,CCA,DCIE,…
Contact: thomas.schend@gmail.com

5. Agenda
4/24/2014 OSDC 20145
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

6. What do we need from IaaS?
4/24/2014 OSDC 20146
Reliable
Scalable (Backend and VMs)
Easy to manage
Long lived workloads, not all VMs are EC2 instances ;-)

7. “VMs are not cattle, they are pets”
Vangelis Koukis, Technical Lead, Synnefo
Quote
4/24/2014 OSDC 20147

8. Agenda
4/24/2014 OSDC 20148
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

9. Synnefo
4/24/2014 OSDC 20149
Open Source
Production-quality IaaS
made for persistent VMs
commodity hardware – No SAN
Large-scale Installations
simple UI for the end-user
uses Ganeti as a Cluster Manager

10. Synnefo History
4/24/2014 OSDC 201410
Design started late 2010
Production since July 2011
~okeanos* public cloud running currently ~8700 VMs
*http://okeanos.grnet.gr

11. Synnefo Layers
4/24/2014 OSDC 201411
Image provided by GRNET

12. Synnefo birdseye view
4/24/2014 OSDC 201412
Image provided by GRNET

13. Synnefo
4/24/2014 OSDC 201413
Build with Ganeti as a backend
Astakos: Identity/Account services
Cyclades: Compute/Network/Image/Volume services
Pithos: File/Object Storage service
snf-manage commandline tool
Basic helpdesk tool
snf-burnin / snf-burnin-run
kamaki commandline client
Code on github

14. Agenda
4/24/2014 OSDC 201414
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

15. Synnefo Architecture I
4/24/2014 OSDC 201415
Image provided by GRNET

16. Synnefo features - Astakos
4/24/2014 OSDC 201416
Identity service
Provides Dashboard
Track Resource usage
Project Management
Multiple Auth methods
Twitter / Google / LinkedIn / LDAP / Active-directory

17. Synnefo features - Cyclades
4/24/2014 OSDC 201417
Network management
L3 Routed/L2 VLANS/L2 isolation with mac filter
nfdhcpd
Archipelago storage abstraction
ceph/NFS/Gluster backend
very fast, thin cloning
Snapshots

18. Synnefo features - Pithos
4/24/2014 OSDC 201418
Object storage service
Efficient syncing
Hashing before upload
Deduplication in backend
Web UI written in GWT
Clients for Windows, Macos and IOS
Sharing public and private

19. Synnefo / Ganeti Communication
4/24/2014 OSDC 201419
Image provided by GRNET

20. written in python / django
apache/gunicorn
postgres database
snf-gnt-eventd / snf-progress-mon
rabbitmq
snf-dispatcher
snf-vncauthproxy
build for wheezy, can run fully virtualized
speaks openstack API but does not share any code
Synnefo Architecture II
4/24/2014 OSDC 201420

21. Secure image deployment of untrusted images
Isolated from the ganeti node
Supports Linux,Windows and BSD
customization tasks
Assign hostname
Delete SSH keys
Change password
Resize Filesystem
Run unattended install (windows)
Inject files
Synnefo - snf-image
4/24/2014 OSDC 201422

22. Synnefo snf-image deployment process
4/24/2014 Central IT Services23
gnt-node evacuate -I hail node3
ImageprovidedbyGRNET

23. Agenda
4/24/2014 OSDC 201424
Introduction
What we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

24. Ganeti History
4/24/2014 OSDC 201425
Very mature, production ready
opensourced since August 30, 2007 by google
uses Mature open source tools LVM,DRBD,python,haskell
Different hypervisors (Xen,KVM,LXC)
Low overhead for cluster management

25. Ganeti Usage at google
4/24/2014 OSDC 201426
Usage at google
small Clusters in Offices (OIB)
large Clusters in Datacenters for virtual desktops
mostly using XEN
Not used for user facing services

26. Ganeti features I
4/24/2014 Central IT Services27
Scale from 1-200 nodes and beyond
Live migration without shared storage
Noderoles: master,master candidate,regular nodes
Nodegroups
OVF import tool
Backup with included tool
Job queue
Commands start with gnt-

27. Ganeti features II
4/24/2014 OSDC 201428
OS Interface to provision new instances
snf-image secure image deployment
ganeti-instance-debootstrap, image, ganeti-os-defs
Htools to balance clusters, plan capacity and do
maintenance
supported file,plain,sharedfile,ceph,NFS,Gluster
Ext storage provider with plugins for
Archipelago, HP EVA,,IBM Storwize

28. Ganeti advanced features
4/24/2014 OSDC 201429
Hooks for every action (pre/post)
Tags for every object
Exclusion tags to keep machines apart from each other
Deploy flash in every node for high performance
(flashcache,bcache,dm-cache)
OOB management interface
Ganeti watcher

29. Ganeti advantages
4/24/2014 OSDC 201430
No need for special hardware (SAN)
But SAN/Shared Storage is supported
Manage cluster as an entity not single nodes (libvirt)
Few dependencies
No database
Easy recovery in case of hardware outage
No SPOF which can affect the reliability of VMs
Very good documentation

30. Deployment automation puppet,chef,cfengine,..
Monitoring icinga/nagios/check_mk/…
Openvswitch LACP + 10GBe
Autorepair
Packages for all major OS debian,ubuntu,rhel/centos
Ganeti best practices
4/24/2014 OSDC 201431

31. Ganeti management interfaces
4/24/2014 OSDC 201432
cli with bash completion
LUXI
RAPI
Different web UIs using the RAPI
Ganeti web manager
Ganetimgr
Synnefo

32. Ganeti quick command reference
4/24/2014 OSDC 201433
gnt-cluster init / info / verify / command / copyfile
gnt-node add / list / failover / evacuate
gnt-instance add / info / replace-disks
gnt-job list …
gnt-network …
gnt-backup …
…

33. Ganeti htools - allocation and placement
4/24/2014 OSDC 201434
written in haskell
hail
hbal
hspace
harep
hinfo
hroller
…

34. Typical Ganeti Clsuter
4/24/2014 OSDC 201435
Node01 Node02 Node03
gnt-cluster init [-s ip] --enabled-hypervisors=kvm cluster
gnt-node add [-s ip] node2
gnt-node add [-s ip] node3
gnt-cluster verify
gnt-instance add -t drbd -I hail -o debootstrap+default vm
Veeamvirtualizationstencils

35. Ganeti Failover and Repair
4/24/2014 OSDC 201436
gnt-node modify -O yes node01
gnt-node failover --ignore-consistency node01
gnt-node evacuate -I hail node01
Node01 Node02 Node03
Veeamvirtualizationstencils

36. “Ganeti didn't have any generally known meaning
before this project. And now it means kick-ass
cluster management”
Martin Beauchamp, on the Ganeti Users List
Quote
4/24/2014 OSDC 201437

37. Agenda
4/24/2014 OSDC 201438
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

38. Login to Astakos
syncing and Pithos UI and App
Create VMs with Cyclades UI
Snapshot Feature
OOB Console
Hotplug of NICs and Networks
Floating IPs
Clean-up
Demo
4/24/2014 OSDC 201439

39. Agenda
4/24/2014 OSDC 201440
Introduction
What do we need from IaaS?
Introducing to Synnefo
Components
Ganeti
Demo
Q & A

40. http://www.synnefo.org
https://github.com/grnet/synnefo
https://code.google.com/p/ganeti/
Try it!
4/24/2014 OSDC 201441

41. 4/24/2014 OSDC 201442
Q & A