SlideShare a Scribd company logo
1 of 39
Download to read offline
Australia’s #1 horse racing website
Our Brands

PuntersParadise.com.au

BestOdds.com.au

+ over 110 individual content websites

rleague.com
Stupid shit we do
More stupid shit we do
Why AWS?
Normal 30 days
12 month traffic
PuntersParadise.com.au

Melbourne Cup Day 2012
•  13,350	
  concurrent	
  users	
  
peak	
  load	
  
	
  
•  550,000	
  uniques	
  in	
  a	
  few	
  
hours	
  
	
  
•  Almost	
  3	
  x	
  more	
  visitors	
  
than	
  Flemington	
  
racecourse	
  
•  Residents	
  of	
  Darwin	
  loose	
  
internet	
  	
  
PuntersParadise.com.au

Our year on year growth

Visits	
  
2009	
  

572,000	
  

2010	
  

796,000	
  

2011	
  

1,731,000	
  

2012	
  

3,008,000	
  

2013	
  

3M	
  +	
  so	
  far	
  
Melbourne Cup Uptime History

2008
2009
2010
2011
2012

Migrated to AWS
about here
The early days.
Immediate wins

We killed our pets and started producing cattle
Automated builds
Performance issues were obvious
Tech was there

Survived 1st Melbourne Cup!
Our Basic Stack

ELB
Cache Servers
ELB
Web Servers

ElastiCache
RDS
Odds Comparison
Our Basic Stack

ELB
Cache Servers
ELB
Web Servers

ElastiCache
RDS

Feed Server
On any given day..

8-10 races per meeting
On any given day..

Average of 12 race meetings per day
On any given day..

+ 45 sporting games
On any given day..

+ 8 greyhound meetings x 10 races
On any given day..

x2 bet types (win/place or sport margin/outrights)
On any given day..

x12 bookmakers!
On any given day..

Odds can change every few seconds
Two problems here.

Processing

Delivery
Our first attempt

while(true)
{
//some magic in here
$market = getMarketInQueue();
//web requests to XML/JSON feeds
$market->hitUpBookmakers();
//save to DB
$market->saveOddsToMySQL();
}
How it works

//this sucks. Doesn’t feel right.
$market->saveOddsToMySQL();
Push that puppy

while(true)
{
//some magic in here
$market = getMarketInQueue();
//web requests to XML/JSON feeds
$market->hitUpBookmakers();
//ahhh. That feels better
$market->saveOddsToDynamoDB();
//like a boss
$market->pushOddsToPusher();
}
Push that puppy
"2081715-Average-FixedWin": {
"2081715-Average-FixedWin": {
"sid": 2081715,
"sid": 2081715,
"bt": "2081715-Average-FixedWin": {
"FixedWin",
"bt": "FixedWin",
"b": "Average",
"sid": 2081715,
"2081715-Average-FixedWin": {
"f": [ "b": "Average", "FixedWin",
"bt":
"sid": 2081715,
"f": [
[
"b": "Average","FixedWin",
"bt":
[
1381882761,
"f": [
"b": "Average",
1381882761,
1
[
"f": [
1
]
1381882761,
[
]
],
1
1381882761,
],
"p": null,
]
1
"p": null,
"ps": null
],
]
"ps": null
"p": null,
],
"ps": null
"p": null,
"ps": null

From
DynamoDB

Pushed
Message

Template
language
Our Full Stack

ELB
Cache Servers
ELB
Web Servers

DynamoDB

Pusher.com

ElastiCache
RDS

Feed Servers
Whats next?

ELB
Cache Servers
ELB

Autoscale!

Web Servers

DynamoDB

Pusher.com

ElastiCache

Autoscale!
RDS

Feed Servers
AWS Reference Architecture

Too easy. Let’s just do that..
Autoscaling the feedserver
Controller

Feed Server

Feed Server
Autoscaling the feedserver
Controller

Feed Server

Feed Server
Autoscaling the feedserver

Feed Server

Feed Server

Feed Server
Autoscaling the feedserver

Feed Server 1

Feed Server 2

Feed Server 3
Autoscaling the feedserver
Autoscaling the feedserver
Autoscaling the feedserver
Our advice

Let everything get hit by a bus
Empower the worker, not the boss
Utilise the AWS API as much as possible
It’s easy. We can all do it.
One last cool thing…

More Related Content

Similar to Architecting high traffic websites with rapid changes in data on AWS

#NewMeetup Performance
#NewMeetup Performance#NewMeetup Performance
#NewMeetup PerformanceJustin Cataldo
 
WebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerWebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerElixir Club
 
MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projectsDmitriy Dumanskiy
 
Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Abhishek Singh
 
TPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxTPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxJirat Kijlerdpornpailoj
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsSerge Smetana
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBRob Tweed
 
Using Apache Solr
Using Apache SolrUsing Apache Solr
Using Apache Solrpittaya
 
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupWeb Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupJonathan Klein
 
Immutability
ImmutabilityImmutability
Immutabilitybyanjati
 
Socket applications
Socket applicationsSocket applications
Socket applicationsJoão Moura
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud
 
Scale, baby, scale
Scale, baby, scaleScale, baby, scale
Scale, baby, scaleJulien SIMON
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien SimonTheFamily
 
Innovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringInnovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringCary Millsap
 
Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Fwdays
 

Similar to Architecting high traffic websites with rapid changes in data on AWS (20)

#NewMeetup Performance
#NewMeetup Performance#NewMeetup Performance
#NewMeetup Performance
 
WebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerWebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan Wintermeyer
 
MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projects
 
ApacheCon 2005
ApacheCon 2005ApacheCon 2005
ApacheCon 2005
 
Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up
 
TPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxTPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and Flux
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDB
 
NoSQL and CouchDB
NoSQL and CouchDBNoSQL and CouchDB
NoSQL and CouchDB
 
Using Apache Solr
Using Apache SolrUsing Apache Solr
Using Apache Solr
 
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupWeb Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
 
Immutability
ImmutabilityImmutability
Immutability
 
Figaro
FigaroFigaro
Figaro
 
Socket applications
Socket applicationsSocket applications
Socket applications
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
 
Scale, baby, scale
Scale, baby, scaleScale, baby, scale
Scale, baby, scale
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon
 
Innovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringInnovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and Monitoring
 
Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"
 

Recently uploaded

[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 

Recently uploaded (20)

[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 

Architecting high traffic websites with rapid changes in data on AWS