A introduction to Packer and how this tool can provide you a way to standardise development environments.

1. STANDARDISE DEVELOPMENT ENVIRONMENTS
AND MACHINE IMAGES WITH PACKER
Marcelo Pinheiro
http://salizzar.net - @salizzar

2. SUMMARY
• Motivation
• Installation
• How it works
• InsideTemplates
• Some examples
• Our experience
• FAQ

3. MOTIVATION
• How to take control inside the
following environment issues?
• Common workstation problems
(HD failure, dead computer)
• Different OS’s
• Extra machine configuration to
enable devs to work (programming
languages, databases, plugins, etc),
taking one or two days to be
ready-to-code
• “Works on my machine” syndrome

4. MOTIVATION
• Vagrant or Docker. Period.
• Embrace virtualisation
• Each offers a way to pre-setup yourVM with necessary libraries,
databases and so on
• No more development databases in your pre-staging DB server,
additional dependencies
• A try to make development environment more similar to production

5. MOTIVATION
• It sounds good, but… how to take control over each applicationVM?
• Sometimes your team needs to use some tools that are not available in
official package repository (or are too old), forcing to manual configuration
after up aVM
• Even running aVM, developer personal choices can contaminate the
application (example: rspec add-ons, irb plugins)
• Some developers don’t have knowledge about Chef / Puppet recipes
• How to maintainVagrant Custom Boxes / Docker Custom Images when
you need to add / change tools, repositories or configs?

6. MOTIVATION
• For sysadms / sysops:
• How to export a new machine image to your
virtualisation server (KVM, Xen,VMWare, etc)
when a new OS release is launched without
“dist-upgrade”?
• How to automate it?

7. MOTIVATION
• Packer for the rescue
• Written in Go
• Owner: Mitchell Hashimoto
(Vagrant, Serf)
• http://www.packer.io

8. INSTALLATION
• http://www.packer.io/downloads.html
• Download zipped binaries for your OS
• OSX
• Linux
• Windows
• FreeBSD
• OpenBSD

9. INSTALLATION
• Move binaries to your /usr/local/bin, ~/bin,
whatever
• It’s done.

10. HOW IT WORKS
• Packer recipes are JSON files
• Validate template:
• $ packer validate your_recipe.json
• Run template:
• $ packer run your_recipe.json

11. INSIDETEMPLATES
• Packer templates have the following structure:
• Variables
• Builders
• Provisioners
• Post-processors

12. INSIDETEMPLATES:
VARIABLES
• User-defined variables to be used along template
• Can be declared in a custom file

13. INSIDETEMPLATES:
VARIABLES
"variables": {!
"box_ostype": "centos",!
"box_osversion": "6.5",!
"box_nick": "6.5",!
"box_arch": "x64",!
"box_type": "base",!
"box_memory": "512",!
"box_cpus": "1",!
!
"iso_arch": "x86_64",!
"iso_type": "netinstall",!
"iso_md5": "939fd1d87c11ffe7795324438b85adfb",!
!
"ssh_user": "vagrant",!
"ssh_pass": "vagrant",!
"hostname": "vagrant-centos-6.5",!
"domain": "vagrantup.com"!
}

14. INSIDETEMPLATES:
VARIABLES
# my_variables.json!
{!
"type": "vmware-iso",!
"vm_name": "mybox-vmw“,!
"guest_os_type": “centos",!
"disk_size": “4096”,!
!
(… other definitions here…)!
}!

15. INSIDETEMPLATES:
BUILDERS
• Create a machine image from scratch
• Download a ISO from official OS mirror, select a
base image to start
• Set CPU cores, memory size, disk size
• See documentation for further details (a lot of
options)

16. INSIDETEMPLATES:
BUILDERS
"builders": [!
{!
"type": "virtualbox-iso",!
"iso_url": "http://an.repository.com/an-image.iso",!
"iso_checksum": "an-checksum",!
"iso_checksum_type": "md5",!
"http_directory": "http",!
"ssh_username": "root",!
"ssh_password": "apassword",!
"ssh_wait_timeout": "100000s",!
"shutdown_command": "echo {{ user `ssh_user` }} | sudo halt -p",!
!
"boot_command": [!
"<esc> ",!
"install ",!
"auto “,!
!
(… other definitions here …)!
!
"<enter><wait>"!
]!
}!
]

17. INSIDETEMPLATES:
BUILDERS
• How to automate setup mundane tasks?
• Minimal set of packages
• Disk partition
• Network
• Timezone

18. INSIDETEMPLATES:
BUILDERS
• For CentOS: Kickstart
• For Debian: Preseed
• For Windows:
• Windows Automated Installation Kit (AIK)
• Microsoft DeploymentToolkit (MDT)

19. INSIDETEMPLATES:
BUILDERS
• Available builders:
• QEMU - KVM and Xen
(experimental)
• VMWare
• Virtualbox
• Docker
!
• OpenStack
• Google Compute
Engine
• Amazon EC2
• Digital Ocean

20. INSIDETEMPLATES:
BUILDERS /VMWARE &VIRTUALBOX
• VMWare:
• vmware-iso: create from scratch
• vmware-vmx: create from a baseVMX file
• Virtualbox:
• virtualbox-iso: create from scratch
• virtualbox-ovf: create from a base OVF file

21. INSIDETEMPLATES:
BUILDERS / QEMU
• Create KVM / Xen images from scratch
• Packer depends on qemu-system-x86_64, available
only on Debian at this time as a binary
• CentOS have qemu-kvm, but you need to
manually override all Packer default options

22. INSIDETEMPLATES:
BUILDERS / DOCKER
• Creates a Docker image by pulling a existent,
starting a container, provision it and exports a .tar
file
• Provision without Dockerfile

23. INSIDETEMPLATES:
BUILDERS / OTHERS
• For other builders, you simply need to inform:
• username / password,API key
• base image
• zone and other related information
• See Packer documentation

24. INSIDETEMPLATES:
PROVISIONERS
• After the setup of a machine image, it’s time to configure
it
• Here is where magic happens:
• Add packages, useful scripts
• Standardise config files
• Apply existent recipes from a CM

25. INSIDETEMPLATES:
PROVISIONERS
• Available provisioners:
• Shell Scripts
• File Uploads
• Ansible
• Chef Solo
• Puppet
• Salt

26. INSIDETEMPLATES:
PROVISIONERS / SHELL SCRIPTS
• Most simple way to setup machine
• Run apt-get, yum and friends

27. INSIDETEMPLATES:
PROVISIONERS / SHELL SCRIPTS
"provisioners": [!
{!
"type": "shell",!
"execute_command": "echo 'root' | sh '{{ .Path }}'",!
"scripts": [!
"scripts/locale.sh",!
"scripts/elrepo.sh"!
]!
},!
{!
"type": "shell",!
"pause_before": "30s",!
"execute_command": "echo 'root' | sh '{{ .Path }}'",!
"scripts": [!
"scripts/vagrant.sh",!
"scripts/sudoers.sh"!
]!
}!
]

28. INSIDETEMPLATES:
PROVISIONERS / FILE UPLOADS
• Need to set default configuration files or upload
some custom packages (.tar, .deb / .rpm) to be
installed later?
• Upload them and after process with a shell script
or CM recipe

29. INSIDETEMPLATES:
PROVISIONERS / FILE UPLOADS
"provisioners": [!
{!
"type": "shell",!
"execute_command": "echo 'root' | sh '{{ .Path }}'",!
"scripts": [!
"scripts/lamp/vagrant.sh",!
"scripts/lamp/apache2.sh",!
"scripts/lamp/php5.sh",!
"scripts/lamp/mysql.sh"!
]!
},!
{!
"type": "file",!
"source": "files/lamp-vagrant/vhost",!
"destination": "/etc/apache2/sites-available/lamp-php"!
},!
{!
"type": "shell",!
"execute_command": "echo 'root' | sh '{{ .Path }}'",!
"script": "scripts/lamp/enable-vhost"!
}!
]

30. INSIDETEMPLATES:
PROVISIONERS / OTHERS
• The following provisioners requires installation before run:
• Ansible
• Puppet
• Salt
• Chef Solo is installed by Packer if not present
• At this time, all provisioners are executed in client mode (no remote
server)

31. INSIDETEMPLATES:
PROVISIONERS / OTHERS
"provisioners": [!
{!
"type": "ansible-local",!
"playbook_file": "recipes/ansible/lamp.yml"!
},!
{!
"type": "chef-solo",!
"cookbook_paths": [ "recipes/chef/lamp" ]!
},!
{!
"type": "puppet-masterless",!
"manifest_file": "recipes/puppet/lamp"!
},!
{!
"type": "salt-masterless",!
"local_state_tree": "recipes/salt/lamp"!
}!
]

32. INSIDETEMPLATES:
POST-PROCESSORS
• After create / setup a machine image, you can:
• Convert to aVagrant Custom Box
• Locally add it as a Docker container
• Publish in a Docker registry
• Publish in a vSphere endpoint

33. INSIDETEMPLATES:
POST-PROCESSORS /VAGRANT
• Defines a box output name
• You can attach aVagrantfile template and other
template files (cookbooks)
• Change compression rate if you want

34. INSIDETEMPLATES:
POST-PROCESSORS /VAGRANT
"post-processors": [!
{!
"type": "vagrant",!
“output": "lamp-vagrant.box"!
}!
]

35. INSIDETEMPLATES:
POST-PROCESSORS / DOCKER
• You can locally import a Docker image
• You can push a Docker image to a registry
• Needs manual login (automated soon)
• Important: Docker pushes a completely new
image, not incremental

36. INSIDETEMPLATES:
POST-PROCESSORS / DOCKER
"post-processors": [!
{!
"type": "docker-import",!
"repository": "salizzar/packer",!
"tag": "0.1"!
},!
"docker-push"!
]

37. INSIDETEMPLATES:
POST-PROCESSORS /VSPHERE
• Upload to a vSphere endpoint

38. INSIDETEMPLATES:
POST-PROCESSORS /VSPHERE
"post-processors": [!
{!
"type": "vsphere",!
"host": "a-vsphere-host.com",!
"username": "my_user",!
"password": "my_password",!
"cluster": "a-cluster",!
“datacenter": "xyz",!
"datastore": "zyx",!
“resource_pool": "zyx",!
"vm_folder": "images",!
"vm_name": "lamp",!
“vm_network": "staging"!
},!
]

39. SOME EXAMPLES
• It’s time to see some code!
• All examples are available on:
• https://github.com/salizzar/packer-examples

40. OUR EXPERIENCE
Ivan IVVasilyevich (theTerrible)

41. OUR EXPERIENCE
• Prepare to argue (sometimes fight :)
• It’s hard to change development tradition of
premature optimisation, ultra-high performance,
personal choices,“nightly build” syndrome
• Create a culture first

42. OUR EXPERIENCE
• Sometimes the better choice must be autocracy-based
• Use OS package system ASAP (or backport / automate installation if
package not exists)
• Introduce to developers a wisdom to use the same package of
programming language / DB / whatever that runs in production (!)
• If is old, upgrade your app to use a newer version
• The same for tools that “vendorize" your app libraries (maven,
bundler, etc)

43. OUR EXPERIENCE
• Make all applications ready-to-setup-and-run with one command
• Track all dependencies with Dockerfile orVagrant Shell Scripts
• Bash scripts are more easy to setup than 3rd party CM tools
at first time
• Adopt a convention to make all applications more similar as
possible about their structure
• Code generators

44. OUR EXPERIENCE
• Divide to conquer
• Adopt a bottom-up strategy
• Minor systems that are easy to setup
• Minor teams
• Start to apply with more systems and greater teams
• Standardise ASAP

45. OUR EXPERIENCE
• At this time, major systems in Locaweb PaaS areVagrant-ready
• git clone, vagrant up, vagrant ssh
• Docker in development
• Internally created a gem to apply standardisation of Rails apps:
• Packaging (Debian)
• Vagrant
• Packer recipes to createVagrant custom boxes, using our mirrors

46. FAQ
• Questions?
• New recipes available on:
• https://github.com/salizzar/packer-vmware

47. THANKYOU!
:)