DevOps for Drupal presentation given at DrupalCon 2013 in Portland. Promet Source shares secrets for automation and how to make your infrastructure hum.

1. DevOps for Drupal: Why
We Cook with Chef
Wednesday May 22nd
DevOps Track
Beginner

2. Overview: Purpose
Why are we all here?
• Interested in DevOps practices
• Interested in automation tools like Chef
• Interested in using them with Drupal
+

3. Overview: Sharing
How are we going to do this?
• Team style presentation and discussion with
views from both sides - Development and
Operations
• Share our experiences and examples... we
will be keeping it real
• We are NOT doing a tool comparison
+

4. Overview: Learn
What do we hope you will learn?
• Why DevOps matters
• Why automation is a must
• Why you can use Chef to help yourselves,
make life easier, and save time
+

5. For more info on Chef, check out fellow
DevOps track session:
The Joy of Cooking - Whip Up a Drupal
Environment with Chef
• Thursday May 23rd at 2:15pm
• OR 203
• Presented by Opscode's Nathen Harvey
Overview: More on Chef +

6. Who We Are
A software development shop...
• specializing in custom Drupal development,
systems integration, mobile, DevOps, and
24x7 support
• based in Chicago with 30+ team members
worldwide
...and yes we are hiring! Come talk to us!
+

7. Who We Are
Playing the part of:
Moderator and
Product Owner
@ Promet Source:
Director of Products
jay@promethost.com
+
Jay Uhlinger

8. Who We Are
Playing the part of:
Development
@ Promet Source:
Solutions Architect
will@promethost.com
drupal.org: wamilton
twitter/hub: @winmillwill
+
Will Milton

9. Who We Are
Playing the part of:
Operations
@ Promet Source:
Systems Administrator
greg@promethost.com
+
Greg Palmier

10. Audience Quiz Time
How many are Developers? (a Will)
+
CC Image courtesy of Pedro Lozano on Flickr

11. Audience Quiz Time
How many are Sysadmins? (a Greg)
+
CC Image courtesy of Sharyn Morrow on Flickr

12. Audience Quiz Time
How many are both? (changing Prod servers)
+
CC Image courtesy of Arthur Caranta on Flickr

13. Audience Quiz Time
How many are other? (the Product Manager)
+

14. Audience Quiz Time
How many are using automated configuration
management tools and processes now in
your everyday work with Drupal?
How many are using Chef?
+

15. DevOps: What It Is
Key Culture
• Foster a collaborative working relationship between
Development and Operations
Key Concept
• Infrastructure as code
Key Results
• Higher deployment rates
• Better reliability, stability, and resilience of Production
+

16. DevOps: 3 Key Patterns
1. Make environments available early in the
Development process
2. Shorten and amplify feedback loops
3. Create reusable deployment procedures
+

17. DevOps: What It Isn't
It is not a collection of bash, Drush, insert
scripts here, etc.
+
DevOps is not about the tools... they don't fix everything
magically
But DevOps relies on some tools to implement its
principles

18. DevOps: Why Is It Important
One-off environments and complicated manual
builds and deployments = FAILURE
Must automate to meet business demands --
support agile, lean, faster, better...
Every environment is Production to someone
Drupal is not different -- all of these apply
+

19. A Classic Scenario
Jay: Will, you said this was working.
Will: It works in Dev.
Jay: Greg, can you look into why it isn't working
in Prod?
Greg: Ok. (later on) @#$%! Dev and Prod are
not the same!
+

20. Another Classic One
The Blog Post: Installing Drupal on AWS
1. Start wizard
2. 50+ steps later
3. A Drupal install
+

21. Why We Chose Chef
The orange color matches ours
of course!
We already had experience (Marius)
doing DevOps work with Chef in
Bay Area with leading tech companies
We build Drupal applications -- we found Chef
to be a good fit for application configuration
management
+

22. The Operations part...
...out with the old and in with the new
+

23. Our Operations Story
How we started with Chef at Promet
Inconsistency was killing our time
How realistic was automation and configuration
management of all of our stuff
+

24. Wasting Time
You are not in the sudoers file. This incident will
be reported.
$ which git
$
Fatal error: Class 'PDO' not found in
blahblahblah
All of these messages diverted your work
+

25. Manual Config Run Amok
• Dev environments != Prod environments
• Shared Dev/Prod/?? environments were
continually bloated
• Over time modifications amounted to more
inconsistency between environments
• Documentation was lacking
o No records explaining changes
o Constant paper trail "paving"
o No great place for it anyways (what wiki?!?)
• More customizations -> High Fragility
+

26. Typing Faster != Solution
• Runbooks, Bash and SSH
o Process to stand up new infrastructure was done
manually via a "runbook”
• Any configuration time was repeated
extensively
• Environment “discovery” bled time
o Snooping around to analyze how things were initially
stood up and potentially modified
(caching, etc.)
• Endless SSH-ing into servers
+

27. Chef Migration
• In with the New
o 2 dedicated servers
§ VM hosts
o Similar model, consolidated
o Entirely Chef managed
o Individual client Chef-spun Dev instances
• Stand Up, QA, and Cutover
o Virtualize legacy
o Considerations mainly PHP
• Monitoring customizations
• Special clients
o Non-Drupal
o Contractually unique
+

28. Chef Migration
And migrating wasn't a complete nightmare....
+

29. Righting the Ship
• Server configuration in Git
o Mitigated "paper trail"
o Team awareness of configs by notifications
o PHP, Apache, MySQL
• System config changes pushed to Chef for
Server propagation
o No more endless SSH-ing
o No more runbook config for every server
o Undoes "helpful" customizations by others
• Direct between me and Config Mgmt
o Configuration entirely “local” with Vagrant
o Git driven accountability and awareness (--stat)
o `knife` is to Chef as drush is to Drupal
+

30. Client Ownership
• Clients maintain ownership of their assets
o Any platform with a knife plugin
o Legacy apps migrated
o Non-Drupal or Promet "old" clients
• Give us their Keys
o Let us be the Designated Driver
o Developers drop in, get pitted
+

31. Client Ownership
• Excessive customization (not perfect)
o Non-Drupal clients or other work
o More so, use client's Hosted Chef
• Contract flexibility
o No Dysfunctional Marriage
• Honor prior client / hosting relationships
o Help clients where they are
+

32. Things That Are Solved
• User Configs
o .gitconfig
o .tmux.conf
o .ssh/config
o shell customizations
o Not necessary but....doable
• Server Configs
o logwatch
o mail
o mysql
o apache
o ssh
o automysqlbackup
+

33. Sysadmin Transformation
Have time to explore/make new tools
Do support in a consistent fashion
Automate all the Things!
"Grow" your team and infrastructure not with
numbers but with talent
We do more work with less resources!
+

34. Sysadmin Team Evolution
We even spun-off a Startup company for
infrastructure and applications automation
and scaling using Chef in the Bay Area (non-
Drupal)
I don't care what you automate with... pick one
and do it!
+

35. The Dev part...
...it's not what you think
+

36. What I Thought I Would Say
• What it was like...
• How it sucked for Devs
• How we changed it
• How it sucked in new ways
• How that lead to the new thing
• Repeat....
• Where we are now
• What's really fun about this for Devs
+

37. The Problem
• We went through too much shit
• It's not even fun to talk about
• It's probably boring
• Someone else already said what I was trying
to justify
• You're probably the choir if you're here
+

38. Drupal: OH @crell
"If you're not using Features, you are not doing
Drupal professionally"
Also:
• If you don't have a one step build, you aren't
using Features correctly
=> Sucking at doing Drupal professionally
• Configuration module is also probably fine
• It's going into core
+

39. So Now My Presentation Is
• What we're doing now
• Why you shouldn't NOT do what we're doing
• Actual examples of how it has helped
• Why Chef is really awesome
• A surprise
+

40. What We're Doing Now
• Everything you need in the repo
o Any scripts or scripting examples
o README files
o (small(-ish)) csv's/xml/etc
o ...pretty much any text file that is not too large and is
useful
• ...Still need a way to automate sourcing
archives of files and sql...
• Vagrantfile + Berksfile
+

41. Berks What?
+

42. +

43. Kirkberks? +

44. What a Dev Has to Do on
Our Team
• Install git somehow
• Install the vagrant package
• Install the vagrant-berkshelf plugin
• git clone/pull && cd-to-the-root
• vagrant up (or reload)
• edit the hosts file or use dnsmasq or similar
+

45. Some Concrete Examples of
This Helping Us Do Better
• Audits
o http://reload.github.io/phing-drupal-template
o Not perfect...
o But now no one has to repeat what is automated
• Every single deployment ever since
o Your Production push should not be the first time
you attempt sync'ing to a Production environment
o ...unless you crave excitement
• Last minute accommodations
o Ever test RewriteBase?
+

46. Add the .htaccess
directive... +
...change the docroot in code!

47. Potential Examples: You
Can't Prove the Negative
• PECL/Pear packages
o Shame on professionals who can't figure it out...
o ...but shame on me for betting the project
• Anything to do with email ever since ever
• Anything to do with external services
o ...memcached
o ...SOLR
o ...even MySQL
+

48. Why You Shouldn't NOT Do
As We Do
Challenges you can deal with, excuses
that are bullshit
On stupid things people say re: vagrant and
chef
• We make great docs!
• My native environment is fine
• Virtualization consumes resources
• I don't have enough time
+

49. +

50. My Native Environment is
Fine
1. No, it isn't
• Your job is to ship to your downstream
o ...not manicure and fetishize your tools
• You are getting strictly less help than you
could get
o ...even if you're getting a lot of help
o ...even if you're paying for help
• You are keeping yourself from helping
+

51. My Native Environment is
Fine (cont'd)
2.Your native environment is personal
• You may even obsess over it
• You may be completely wrong
• You will be happier without each other
+

52. Virtualization Consumes
Resources +

53. I Don't Have Time...
...to do everything perfectly
• We didn't either, and we're doing far better
than we were
• We do not have a massive continuous
delivery, cloud orchestration and scaling
infrastructure
• ... But now what we have is in code
+

54. Chef is Really Awesome
...Puppet is probably cool too
• Stewardship and leadership
• Constantly evolving ecosystem
• A community hell-bent on sharing
• Kicks ass even if you don't
+

55. So Easy I Can Share It with
You!
Don't try it on this connection
bash <(curl -L -s http://bit.ly/10McZSZ)
github.com/promet/drupal_cookbook
Get to work already!
+

56. Why You Should Use this
Project (or something like it)
We must advance the state of the art for Drupal
deployment and application management
The CMS that is easiest to flexibly deploy will
be the CMS of the future
All the features and D8 config exports in the
world can't help you if you can't automate it
+

57. Other Projects/Strategies
• If you have a dope Jenkins workflow...
o help me automate it with Chef so we can all use it!
• Please use Puppet if it suits you
o and be vocal about hardcoded conventions that
make your work hard
• Kraftwagen - kraftwagen.org
• Ariadne by patcon - github.com/
myplanetdigital/vagrant-ariadne
+

58. The Wrap-Up part...
...what should you remember
+

59. Key Summary Points
Process and team is the focus
• Get your Devs and Sysadmins in the same
room and start communicating
Tools are there to help
• Start automating now
Chef is an awesome tool
• It works and it helps
+

60. Q&A
Ask us questions now
Talk to us afterward
Stop by and see us at Booth 201
Thanks!
+

61. Feedback?
Evaluate session at:
portland2013.drupal.org/schedule