AWS OpsWorks is a solution for managing applications of any scale or complexity on the AWS cloud. Accelerate your use of OpsWorks by learning how to use several of its operational features in this Zero to Sixty session. It starts with a demo of the OpsWorks main workflows—manage and configure instances, create and deploy apps, monitoring, and security. BeachMint will explain how they set up OpsWorks as part of their continuous deployment pipeline. The session finishes off by explaining how to use the OpsWorks API and Chef recipes to automate standard operating procedures. Demos and code samples are available to all session attendees. Are you new to AWS OpsWorks? Get up to speed for this session by first completing the 60-minute Introduction to AWS OpsWorks lab in the Self-Paced Hands-On Lab Lounge. It will lead you through all major functions of the service with a fun example.

1. DMG202 - Zero to Sixty:
AWS OpsWorks
Thomas Metschke, Amazon
November 13th, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. AWS OpsWorks
Helps you to model your entire application from
load balancers to databases starting from
templates for common technologies or building
your own.
You have full control of deployments, scaling,
monitoring, and automation of each component.

3. AWS Application Management Services
Higher-level services
Elastic Beanstalk
Convenience
OpsWorks
Do it yourself
CloudFormation
EC2
Control

4. The Heart of AWS OpsWorks
Agent on each
EC2 instance
talks with
OpsWorks

5. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.

6. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.

7. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.

8. AWS OpsWorks Agent Events
setup
configure
deploy
undeploy
shutdown

9. A Stack

10. A Stack with Layers

11. A Stack with Layers and Instances

12. Enough Talking
DEMO TIME

13. Recap of Demo
•
•
•
•
•
•
Created a new Stack from scratch
Created an app and database layer
Started instances in different Availability Zones
Configured Auto Scaling
Load balanced by Elastic Load Balancing
Deployed an application

14. Continuous Integration

15. John Gaa
VP of Engineering at BeachMint

16. •
•
•
•
Founded in 2011
Next generation social commerce company
Parent to: JewelMint, ShoeMint, StyleMint, & IntiMint
Over 6 million registered customers

17. Problem Set …

18. Our Store Verticals are Powered By
•
•
•
•
•
•
Amazon RDS
Amazon ElastiCache
Amazon SQS
Amazon SNS
Amazon CloudFront
Amazon Redshift
RDS
ElastiCache
SQS
SNS
CloudFront
Redshift

19. • Multiple releases a day
• 1 sysadmin
• 100+ instances between production
and test environments

20. Jenkins & AWS OpsWorks to the Rescue

21. Confidence to Release Every Day
•
•
•
•
Unit tests
Integration tests
Load testing
Manual testing

22. What it Looks Like in AWS OpsWorks
Environments are represented as
stacks
Our different verticals are
represented as separate layers within
each stack

23. Architecture Repeated in Each Environment
IntiMint
ShoeMint
StyleMint
Stack
JavaScript widget
API
Checkout
RDS
Registration
Solr
Product
ElastiCache
Drupal
Social
Drools
Layers
JewelMint

24. The Details

25. AWS OpsWorks in Our CI Processes
• Get instances associated to a layer
• Update code based on gittag version stored in
custom JSON
• Target the instances and run Chef scripts

26. The Glue: Custom JSON
{
"environment":{
"env":"production"
},
"jewelmint":{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net"
}
},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{ "1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }
}
}
"environment":{
"env":"production”
},
Determines what
environment to be built

27. The Glue: Custom JSON
{
"environment":{
"env":"production"
},
"jewelmint":{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net"
}
},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{ "1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }
}
}
"jewelmint"{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
Code version to be
deployed to a layer

28. The Glue: Custom JSON
{
"environment":{
"env":"production"
},
"jewelmint":{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net"
}
},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{ "1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }
}
}
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net" }},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{
"1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }}
Configuration and
attributes of
supporting systems

29. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
//Get the Stack we are updating
$onlineInstanceIds = array();
$allInstances = $opsWorks->describeInstances(array('LayerId' => $layerId))>get("Instances");
foreach($allInstances as $instance) {
if ($instance['Status'] == 'online') {
$onlineInstanceIds[] = $instance['InstanceId'];
}
}
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA

30. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
$deployment = $opsWorks->createDeployment(array(
'StackId' => $stackId,
'Command' => array(
'Name' => 'execute_recipes',
'Args' => array(
'recipes' => $recipe
)
),
'InstanceIds' => $onlineInstanceIds
));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA

31. Load Testing
•
•
•
•
•
Code
Configuration
EC2 instance type
System architecture configuration
Capacity planning

32. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
//Get the Stack’s custom JSON from OpsWorks
$customJsonArray = json_decode($theBuildStack[0]["CustomJson"]);
//Replace the gittag with the new tag
$customJsonArray->{"jewelmint"}->{"gittag"} = "thenewgittag";
//Convert back to string
$modifiedJson = json_encode($customJsonArray);
//Update Stack settings in OpsWorks
$updateRes = $opsWorks->updateStack(array("StackId" => "the-buildstackid","CustomJson" => $modifiedJson));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA

33. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
$deployment = $opsWorks->createDeployment(array(
'StackId' => $stackId,
'Command' => array(
'Name' => 'execute_recipes',
'Args' => array(
'recipes' => $recipe
)
),
'InstanceIds' => $onlineInstanceIds
));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA

34. Path to Production
• QA and Product group determines what goes to
production
• Each release candidate has a tag associated
with it
• QA updates QA and Stage environments using a
tool based on the AWS SDK

35. Thanks!
www.jewelmint.com
www.stylemint.com
www.intimint.com
www.shoemint.com

36. Automate Standard Operating Procedures

37. 3 AM – Alarm goes off

38. Wouldn’t it be nice to have help?

39. The idea: gather and ship logs to Amazon S3
as soon as the CPU load is to high

40. We will use the
following setup

41. AWS OpsWorks stores 1-minute
metrics in CloudWatch

42. Every instance creates an
alarm for high CPU load

43. CloudWatch alarm action:
write to SNS topic

44. SNS publishes to
queue in SQS

45. Watcher instance polls
SQS for notifications

46. On alarm notification,
call OpsWorks API to …

47. Execute a script on the
affected server

48. Logs are gathered and
written to Amazon S3

49. And Again
DEMO TIME

50. Demo Recap
•
•
•
•
Instances set up alarms on CloudWatch
Alarms create notification in SNS
SNS publishes to SQS queue
Watcher instance polls queue and calls AWS
OpsWorks to execute recipe on instance
• The recipe runs a script to upload logs to S3

51. More Information about AWS OpsWorks
•
•
•
•
If not done yet, do the AWS OpsWorks lab!
Find us in the AWS Booth
Follow us on Twitter @AWSOpsWorks
Find us on YouTube
• AWS OpsWorks survey
http://tinyurl.com/OpsWorksSurvey2013

52. Other Talks During re:Invent
DMG304 - AWS OpsWorks Under the Hood
• Jonathan Weiss & Reza Spagnolo
• Thursday, Nov 14, 3:00 PM - 4:00 PM – Murano 3206
DMG305 - How Intuit Leveraged AWS OpsWorks
as the Engine of Our PaaS
• Capen Brinkley & Rick Mendes of Intuit, Inc.
• Thursday, Nov 14, 4:15 PM - 5:15 PM – Murano 3206

53. Please give us your feedback on this
presentation
DMG202
As a thank you, we will select prize
winners daily for completed surveys!

54. Backup Slides

55. Recipes
• Default
Writes Ruby script to the instance and runs it
• create_alarm
Writes a Ruby script to create an alarm and ties it to the right SNS
topic
• send_logs
Writes a Ruby script to the instance that can pack and ship the logs
to S3

56. Watcher Code
queue = AWS::SQS.new.queues["<%= node[:watcher][:sqs][:url] %>"]
queue.poll(:initial_timeout => false) do |msg|
begin
alarm = JSON.parse(msg.body)['Subject'].start_with?('ALARM')
instance_id = JSON.parse((JSON.parse(msg.body)['Message']))['Trigger']
['Dimensions'].first['value']
if alarm
opsworks = AWS::OpsWorks.new.client
deployment = opsworks.create_deployment(
:stack_id => "<%= node[:opsworks][:stack][:id] %>",
:instance_ids => [instance_id],
:command => {
:name => 'execute_recipes',
:args => {'recipes' => <%= node[:watcher][:execute_recipes] %>}
},) end end end

57. Alarm Creation
id = "<%= node[:opsworks][:instance][:id] %>"
action ="<%= node[:watcher][:sns][:arn] %>"
alarm = AWS::CloudWatch.new.alarms.create("#{id}_cpu_idle", {
:namespace => 'AWS/OpsWorks',
:metric_name => 'cpu_idle',
:dimensions => [{:name => 'InstanceId', :value => id}],
:comparison_operator => 'LessThanOrEqualToThreshold',
:evaluation_periods => 1,
:period => 60,
:statistic => 'Average',
:threshold => 20,
:actions_enabled => true,
:alarm_actions => ["#{action}"],
:alarm_description => 'watching the available CPU on the instance'
})

58. Pack and Send Logs – Generated Ruby Script
timestamp = Time.now.utc.to_i
archive = "/tmp/#{timestamp}.tgz"
source = "/var/log"
s3bucket = "<%= node[:watcher][:s3][:bucket] %>"
object = "<%= node[:opsworks][:stack][:name] %>/<%= node[:opsworks][:instance][:hostname]
%>/#{timestamp}.tgz"
`cd #{File.dirname(source)} && tar czf #{archive} #{File.basename(source)}`
s3 = AWS::S3.new.buckets[s3bucket].objects[object].write(:file => archive)