Wonga has been around for many years and it has gone through massive growth periods in the different regions they operate. As a result, each engineering team did their best to keep going but that approach is not sustainable when you want to create an automated platform that should allow you to easily launch new financial products anywhere in the globe. Since the beginning of 2016, we have been revamping how we automate things at Wonga, trying to set the foundations for such a platform, with emphasis on simple, pragmatic and efficient tooling that allows us to operate in a hybrid infrastructure (AWS and VMWare Vsphere) with both Windows and Linux servers. There is still a long way ahead but this talk will be about where we were, where we want to be, tactical steps to start the journey and why we chose our current tools.

1. FROM SNOWFLAKES TO A COMMON
AUTOMATED PLATFORM
Ricard Clau
John Paul Newman

2. WHAT DO WE DO?
• Automation group function in Wonga
• Small team servicing X engineers in 5 locations
• CI / CD pipelines, Logging / Monitoring,
infrastructure provisioning, config management, …
• Most of the team is quite new to the company

3. A BIT OF WONGA HISTORY
• Started in 2007, DevOps was not even a thing!
• Regions expansion, acquisitions, …
• Massive growth, engineers did their best to keep up
• Regulations happened, FCA approval needed
• Massive turnover, knowledge lost, lack of docs…

4. CURRENT PROBLEMS
• Snowflake servers, many attempts in the past failed
• No unified processes in the group
• Not great monitoring dashboards
• No DevOps culture, we are seen as a service team
• Sometimes, all these are great excuses

5. WANTTO ACHIEVE
• Build / Provision servers & infrastructure from code
• Needs to work for both Windows and Linux
• Hybrid cloud (AWS) / datacenter (vSphere)
• Simple, pragmatic and efficient tools
• Progressive introduction of new tooling

6. INITIAL ROADMAP
• Pick tools to build / automate everything
• Rationalise CI / CD tooling
• Plan a progressive migration to the AWS cloud
• Rationalise logging / monitoring infrastructure
• Build platform capabilities that can be shared

7. CI
Jenkins Team City ThoughtWorks GoCD

8. CI
Jenkins Team City ThoughtWorks GoCD

9. JENKINS
• Hundreds of plugins and documentation
• Job configuration from code via Jenkins
Job Builder (JJB) or Wonga's own JJB Ruby DSL*
• Free! Allowing each team to have their own self-
managed server and agents
* https://github.com/wongatech/wongatech.github.io/blob/master/_drafts/jenkins-job-builder-ruby-dsl.md

10. SCM
Gerrit GitLab GitHub

11. SCM
Gerrit GitLab GitHub

12. GERRIT
• Review and CI validation processes
• Supports replication for DR
• LDAP backed authentication
• Integrates with internal tools, like JIRA
• Detailed ACLs and nice project structure

13. GITHUB
• Nice UI and developers familiarity
• Hooks integration
• Debatable Pull Requests model
• Delegate DR, HA, etc… to Github
• Has source code based wiki

14. SCM
Puppet Labs Opscode Chef Ansible

15. SCM
Puppet Labs Opscode Chef Ansible

16. ANSIBLE STRENGTHS
• Easy learning curve
• Agentless but you can also do ansible-pull
• Plays nicely with running Windows servers
• Decent community roles in Ansible Galaxy

17. ANSIBLE ISSUES
• Ansible 2.0 is still a bit buggy
• You always need a Linux control machine
• Less flexible than Chef or Puppet (unless you
write your own modules…)
• Variable quality in Ansible Galaxy

18. MONITORING

19. MONITORING

20. ELK STACKVS SPLUNK
• Decent in-house Splunk experience
• Splunk dashboards still a bit better than Kibana
• Logstash needs to configure GROK, Splunk can
mostly guess itself
• Still experimenting with ELK for our own stuff

21. INFLUXDATA
• A platform for collecting time-series data
• Model system metrics and business metrics
• We use the Telegraf agent to send metrics,
InfluxDB to store data and Grafana dashboards
• Need to explore Kapacitor for monitoring

22. INFLUXDATA CONCERNS
• Experimental support for Windows
• Still 0.12 at the moment. Breaking API changes
• Many people get confused about time-series data
• InfluxDB cluster not free anymore

23. INFRASTRUCTURE

24. PACKER STRENGTHS
• Works nicely with both Windows and Linux
• Plays nicely with AWS and VMWare
• Easy to share provisioning scripts
• Easier to understand than Config Management
tools (Chef, Puppet or Ansible)

25. PACKER CAVEATS
• Need to be very prescriptive or the number of
templates can get out of hand quickly
• Hard to go with a DRY approach
• Often not much benefit in Linux systems vs
provisioning tools on startup

26. TERRAFORM STRENGTHS
• Plays nicely with AWS and has some initial
support for vSphere (actively developed)
• Has a nice pluggable providers system to
automate virtually everything… if you know Go
• No real cloud agnostic competitor

27. TERRAFORM ISSUES
• Not great documentation and error messages
• Some providers don´t have nice update support
• Tricky to store state files
• Terraform modules are still a bit hacky
• Relatively immature overall

28. SOME SUCCESS!
• Tools decided, good engagement in the team
• Building Packer AMIs andVMWare templates
• Some services already fully managed by Ansible
• Many servers rebuilt from config management
• Small Terraform setups working

29. THE (NEAR) FUTURE
• Consul for Service Discovery and Config storage
• Better secrets / keys management (Vault)
• Start the Prod migration to AWS (some
components already running in PreProd)
• Improve the current successes and think platform

30. BABY STEPS

31. A LONG JOURNEY AHEAD

32. QUESTIONS?
• BTW… incidentally… we are hiring!
• Come talk to us!
• Thank you for listening!