Application design for the cloud using AWS

1. Application Design for the Cloud/AWS
Jonathan Holloway (@jph98)
Application Architect @ Brightpearl

2. A little bit of background
Brightpearl grew out of a company called Lush
Longboards in Bristol
Chris Tanner and Andrew Mulvenna identified
a need for a single online business
management app
First customer on Brightpearl in 2007
Over 1300 customers currently on Brightpearl
and growing
The numbers:
Founded 2007
Today, we have:
1,300+ customers
53 countries
87 employees
(in San Francisco and Bristol)
$1.3 billion
gmv processed

3. What Does Brightpearl Do?
- Multi-channel integration (Amazon, Ebay)
- Storefront integration
- Inventory, orders, and customers
- Logistics and shipping
- Web and POS interface
- App store for third party integrations

4. Business @ Scale on the Amazon Platform
Over 1,300 customers run on our cloud based platform

5. Company Culture
Rapidly growing
Team, Product, Technology, Career
Team and People
Passionate, Diverse, Agile
The way we work
Open Plan, Open Minded, Autonomous
Great social life
We call our colleagues friends and do fun stuff
together outside the office.
Career development
We invest in our people and support their
career aspirations.
Open atmosphere
Our managers don’t have offices and openly
tell us about the business’ progress.
Multi-cultural workforce
We have employees who come from over 20
different countries.

6. Tech Culture
General tech lunch and learns (TLAL) for various technology talks
(Java, PHP, Javascript)
Ruby project workshop for building apps once every two weeks,
encourage use outside development
Friday - half day for personal tech projects, e.g. dashboards,
product improvements
Java Meetup Group for external talks with developers from other
companies in the Bristol and Bath area. We host PHPSW along with
Basekit in Bristol

7. My Background
Application Architect - Solutions/Technical, Developer
Started with SaaS back in 2010 with email archiving and large scale
storage/search solution
- Cassandra, Hadoop, Pig, Lucene and Jersey
Worked in statistical computing with distributed grids (Oracle Grid
Engine, LSF). Deployed on large compute clusters internally in
pharma, looked at cloud based solution using Starcluster, Python
Java background, Ruby, Python and Javascript

8. Cloud, Tech Overview

9. SaaS - Big Product Examples
Lots of big SaaS based products on the web today including:
On-demand movie rental
Customer Relationship Management
Microsoft Office Toolsuite Online
Conference and collaboration

10. SaaS and PaaS and IaaS
It’s all very confusing… think of it as a pyramid
IaaS - Infrastructure as a service (Tools/services for devops)
- Amazon EC2, Windows Azure, Heroku
PaaS - Platform as a service (Tools/services for app devs)
- AWS Beanstalk, Heroku, Google App Engine, Redhat
Openshift
SaaS - Software as a service (Apps for end users - yay)
- e.g. Netflix, Brightpearl, Salesforce, Office365

11. How do I know when a product should be SaaS based?
- Have to be careful with data requirements, data at rest,
transfer because of the public cloud. VPC and VPN can help.
- I/O and throughput might prohibit movement of files
- Have to be careful don’t just move your application into the
cloud… break it down… re-assemble with cloud based
application services
- Factor in availability, performance, failover, reliability. Don’t
underestimate reliability

12. SaaS - Archiving Solution
Customer Account
Metadata
Postgres/Slony
Search Services
SOL
R
SOL
R
SOL
R
Content Extraction
Tika Tika
Content Storage
Solaris/ZFS
Search Interface
Django
Tika
Mail Archive
Mail Archive
Mail Archive
- Worked on this for a private cloud solution
- Why doesn’t it fit the public cloud - Amazon/Rackspace?
- Couldn’t move it “as is”

13. SaaS Distributed Grid - Cloud Bursting Scenario
Oracle Grid Engine
Grid Master
Slave
Slave
Slave
Stats Desktop
SAS
Matlab
Amazon Cloud
Slave
(m3 large)
Slave
(m3 large)
Slave
(m3 large)
Slave
(g2 GPU)
Slave
(g2 GPU)
Slave
(g2 GPU)
1. Provision new instances
2. Extend Capacity Internally
3. Transfer Files
4. Execute Files
5. Shutdown when done
R

14. SaaS eCommerce Business Management Scenario

15. Amazon Web Services - Cloud Computing Services
Using AWS for ~ 4 years as a IaaS platform
Brightpearl is designed for use on AWS
Make use of both US and European datacentres, multi-availability
zones - approx 90 EC2 instances
All Amazon Linux based images, use Centos 5.x in dev/test
We approximate an environment for development and test

16. Architectural Overview
Will break it down into the following views:
- Infrastructure and Operating System
- Application Services (Queuing, Data Storage, Load Balancing)
- Software Stack (Brightpearl Application - JS, PHP and Java)

17. Infrastructure
We build on various Amazon base Images (based on Amazon Linux)
with different specifications...
- m1 medium (webserver)
- t1 micro (mail relay)
- m1 small (messaging)
EC2Instances is useful - http://www.ec2instances.info/
Software is provisioned on top of the base O/S with Chef (we
maintain this configuration and keeping it up to date).

18. Application Services
Elastic Compute Cloud (based off Xen)
- Various instance sizes (small, medium large, xlarge)
- Basis is an AMI - Centos, RHEL, Windows
Amazon RDS (Relational Database Service) - aka MySQL
- Data Storage
Content delivery network, think Akamai
- Global delivery of static resources (images, content)

19. Application Services
Key/value store - useful for storing large data that
won’t fit in a relational database
Amazon S3 - storage service for files
ELB (Elastic Load Balancer) - instance load balancing

20. Languages - Javascript
Javascript for DOM manipulation, data binding, validation
- Functional, oh so functional
- JQuery for DOM manipulation and UI elements
- Backbone for structure (+ CommonJS)
- Mocha for testing
- JS on the serverside - Node
- Dependency Management (Bower, NPM)

21. Languages - PHP
PHP for web development in the presentation tier
- Dynamically typed, interpreted
- Single threaded
- Well supported, lots of third party software
- Lightweight, fast and proven

22. Languages - Java
Java for scaling services out
- Statically and strongly typed
- Good for concurrency and parallelisation
- Good library, framework & IDE support (Intellij)
- Build RESTful API’s for PHP to communicate with

23. Languages - Ruby
Ruby - for provisioning infrastructure, configuration and test
- Our “devops” and “test engineer” language
- Dynamically typed, multi paradigm
- Readable, testable, way cool
- Great third party library support
- Chef by Opscode used for configuring EC2 instances
- Cucumber and Webdriver for application testing

24. Design Problems

25. AWS Design Considerations
Sometimes we need to be agnostic for performance, cost reasons
and also for vendor (Amazon) lockin
Have to build in failover to each individual service
Don’t use SQS (Simple Queueing Service) - instead we roll our own
Roll our own datagrid for cross EC-2 instance data - an in-memory
datagrid. Think distributed Java collections.
Make use of a distributed file system for transient file storage

26. Problems at Scale
Design for redundancy:
- Ephemeral storage by default. Use EBS (Elastic Block Storage)
- Multiple copies of services deployed on instances
- Multiple instances for failover
Design for scale:
- Partitioning of accounts using separate RDS instances

27. Problems at Scale
Design for concurrency:
- Immutability is key, Actor Model, STM are useful
Design for performance:
- Profile everything (Yourkit is great for this)
- Java is very memory hungry… tune the JVM and GC strategy

28. Problems at Scale
Design for cost:
- Reserved instances (up-front cost) can save a fair bit
Design for content delivery:
- Server side caching (Varnish, mod_cache)
- Client side caching (expires headers, etags)
- Content Delivery Network - (Cloudfront, Akamai)

29. Design for failure:
- Internal Services (Redundant Copies)
- External Services
- Protect against them overloading your internal
services
- Don’t flood them with your traffic
Problems at Scale

30. Future Tech
Some future technology we’re interested in...
- Web Components (Polymer)
- Functional JVM languages (JDK 8 Streams, Clojure)
- Docker (LXC containers) for virtualisation
- Interested in Quasar for lightweight
threading

31. Oh by the way… we’re expanding...
Current Open Positions
Senior Developers
Graduate Developers
Test Engineers
Key Events @ Brightpearl
5th August - Java Meetup Group
20th August - PHPSW Meetup
Questions?