I had a great audience at the Illinois Java Uses Group at Nielsen in Schaumburg for this presentation. It is about Tvarit, a maven plugin that can fit nicely in a continuous delivery pipeline for the AWS platform. Feedback, comments and questions are all very welcome! Thank you. Source and instruction are on github: https://github.com/sdole/tvarit-maven

1. AWS SDK
DevOps Automation
Sachin Dole

2. About me
▪ AWS Certified Solutions Architect
▪ Independent AWS Consultant – Genvega Inc.
▪ Trelair.com – now defunct startup
▪ www.cricclubs.com – ½ million+ page views per month
▪ Backup & Archival on S3 – project for law firm
▪ EC2 automation for image processing
▪ Available for AWS consulting – find me: www.genvega.com

3. About CricClubs (My customer)
30th Jan
2016
@2015 CRICCLUBS.COM | www.cricclubs,.com | CONFIDENTIAL 3
Started in
2008
Geographic Usage
20+ Countries
600+ Leagues 100,000+
Players
Headquarters
Milwaukee, WI,
USA
Features
50+ features on
Website and iOS,
Android, Windows
Mobile Apps
Premium Support
24/7 via E-mail, Phone and
Ticketing Systems
Data Centers
High Availability with
servers across 12 Data
Centers
4 million pageviews/month

4. About you?
▪ Have day to day AWS experience?
▪ Have personal projects on AWS?
▪ Want a good introduction to DevOps & AWS?

5. Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services

6. Topics
▪ DevOps for Trelair
▪ Lessons learnt
▪ Tvarit.io & AWS Java SDK
▪ Questions?

7. Trelair.com

8. System Architecture
Clients
ELB
Wildfly on EC2
S3 & Postgres
External
Services

9. Building Trelair
Commit
Build
FTP
Deploy
Test
Backup
&
Release
1 to 8 person-hrs
DML Disk Compute DB App server Backups

10. Commit
Build
FTP
Deploy
Test
Release
Enterprises?
1 to 8 person years
DML Disk Compute DB App server Backups
Dev
EA
Test Ops
EA

11. Trelair Automation
GitCommit
• Teamcity
• JBoss
DMR
Auto-
Deploy
Selenium
Test &
Release
DevOps + 0 person hours
Custom Maven Plugin
Used AWS Java SDK
AMI DML/DDL DNS Snapshots Scaling
Auto

12. Automated Flow
A. Commit Code (the only manual step)
B. CI build
C. DB DML patches are applied to replica
D. Deploy code to candidate server
E. UI tests run
F. DML patches applied to production
G. Candidate server is added to cluster
H. Old servers are terminated

13. AWS API
us - east
AWS Java
SDK
JBoss DMR Wildfly AS
Provision
Deploy

14. Lessons
▪ Automation is easy – mindset is difficult
▪ Better to automate in small increments – big bang is a waste
▪ Automate everything – partial automation is overhead
▪ Release frequently – rely on automation
▪ Use selenium automation (e.g. saucelabs.com)
▪ Rely on AWS products

15. Tvarit
▪ http://www.tvarit.io, https://github.com/sdole/tvarit-maven
▪ Trelair automation on steroids, opensource
▪ Automated provisioning, deployment and scaling
▪ Alpha release available in Maven Central
▪ Free support through GitHub issues
▪ Working α version & a deep roadmap

16. Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services

17. Tvarit – demo
▪ CloudFormation templates
▪ Maven goals
▪ VPC, IAM console
▪ EC2: LC, ASG, ELB
▪ S3 bucket
▪ Cloudwatch monitoring, flow logs
▪ Code walkthrough

18. API structure
▪ Many flavors – Java, C++, JS, Ruby, Python, CLI etc.
▪ APIs are RESTful
▪ Deep documentation

19. API Structure
AmazonEc2Client ec2 =
new AmazonEc2Client(new BasicAwsCredentials());
SomeEc2Request req = new SomeEc2Request();
req.withSomething(thing1).withSomething(attr1);
SomeEc2Result result = ec2.submitRequest(req);
inspectWithYourLogic(result);

20. Automation Security - Individuals
IAM
us - east
us - west
Access
Keys
groups
users
roles
Authorized

21. Automation Security - Services
IAM
Other
services
Other
services
ec2
lambda
S3
Assumed
Role
Authorized

22. AWS Automation Features
▪ Autoscaling
▪ Monitoring
▪ Alarms
▪ Cloudwatch Events
▪ Lambda Functions
▪ S3 Events, SQS and SNS events
▪ SWF
▪ CloudFormation

23. Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDK
Autoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1

24. Other concepts:
Blue-Green Deployment
http://martinfowler.com/bliki/BlueGreenDeployment.html

25. Other concepts:
Canary Release
http://martinfowler.com/bliki/CanaryRelease.html

26. Revisit Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDK
Autoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1

27. Tvarit Release concept (future)
ELB
ELB
Router
App Versions
Router: routes customers based on Http Headers to correct app version

28. Step Back – what is (not) DevOps?
Is…
Infrastructure as code
Not…
Auto-scale, failover, monitoring
Continuous delivery
Dedicated teams, Dev, Test, Ops
Forms, Emails, Release plans
Infrastructure Architects/EA

29. Soft side of DevOps
▪ Cloud-native vs Lift ‘n shift
▪ CALMS
▪ C – Culture
▪ A – Automation
▪ L – Lean
▪ M – Measurement
▪ S – Sharing
▪ Automation as Kaizen – not a single project

30. Costs of AWS Automation
▪ In most cases – FREE
▪ Priced as usual: Compute, Storage, Monitoring, Network
▪ Monitoring is a must
▪ CPU, Memory, Network, Disk…
▪ Health checks, failed hardware, traffic spikes
▪ Audit trail – API actions, modifications
▪ Configuration changes

31. How to improve Tvarit
▪ Open an Issue on GitHub:
“We use a customized version of tomcat – can you make the
app server container configurable so we can use our own?”
 Fork it on GitHub and DIY
 Email me

32. Other things
▪ Chef, Puppet, Ansible, SaltStack
▪ AWS OpsWorks, Beanstalk
▪ AWS CodeCommit, CodeDeploy

34. How to find me
@sdoledotcom
https://linkedin.com/in/sdole
sdole@genvega.com
http://genvega.com/blog/
https://github.com/sdole/

35. Ω
Thank you