How to scale up, out or down in Windows Azure

HOW TO SCALE UP,
OUT OR DOWN IN
WINDOWS AZURE

JUAN DE ABREU
VP, Sales Director

OUTLINE
• SCALABILITY
Achieving linear scale
Scale Up vs. Scale Out in Windows Azure
Choosing VM Sizes
• CACHING
Approaches to caching
Cache storage
• ELASTICITY
Scale out, scale back
Automation of scaling

HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar

A PRIMER ON SCALE
SCALABILITY IS THE ABILITY TO ADD CAPACITY TO A COMPUTING
SYSTEM TO ALLOW IT TO PROCESS MORE WORK


A PRIMER ON SCALABILITY
• VERTICAL SCALE UP • HORIZONTAL SCALE OUT
Add more resources to a single Adding additional computation units and
computation unit i.e. Buy a bigger box having them act in concert
Move a workload to a computation unit Splitting workload across multiple
with more resources computation units
e.g. Windows Azure Storage moving a
partition.


VERTICAL VS. HORIZONTAL
• FOR SMALL SCENARIOS SCALE UP IS CHEAPER
Code ‘just works’
• FOR LARGER SCENARIOS SCALE OUT ONLY SOLUTION
Massive diseconomies of scale
1 x 64 Way Server >>>$$$ 64 x 1 Way Servers.
Shared resource contention becomes a problem
• SCALE OUT OFFERS PROMISE OF LINEAR, INFINITE SCALE


SCALABILITY != PERFORMANCE
OFTEN YOU WILL SACRIFICE RAW SPEED FOR SCALABILITY
FOR EXAMPLE; ASP.NET SESSION STATE


ACHIEVING LINEAR SCALE OUT
• REDUCE OR ELIMINATE SHARED RESOURCES
• MINIMIZE RELIANCE ON TRANSACTIONS OR TRANSACTIONAL TYPE
BEHAVIOUR
• HOMOGENOUS, STATELESS COMPUTATION NODES
We can then use simple work distribution methods, Load balancers, queue distribution
Less reliance on expensive hardware H/A


UNITS OF SCALE
CONSOLIDATION OF ROLES PROVIDES MORE REDUNDANCY FOR SAME
CREATE AS MANY ROLES AS YOU NEED ‘KNOBS’ TO ADJUST SCALE

Clean Up Role Web Site Role’ WCF Role

Loss of an instance results in 50%
capacity loss in web site.

Cache Build Role


UNITS OF SCALE
CONSOLIDATION OF ROLES PROVIDES MORE REDUNDANCY FOR SAME
CREATE AS MANY ROLES AS YOU NEED ‘KNOBS’ TO ADJUST SCALE

Web Driven Role

Loss of an instance results in 25%
capacity loss in web site.

Queue Drive
Role


VM SIZE IN WINDOWS AZURE
• WINDOWS AZURE
Supports Various VM Sizes
~800mb/s NIC shared across machine
Set in Service Definition (*.csdef).
All instances of role will be equi-sized

<WorkerRole name=“myRole" vmsize="ExtraLarge">


Remember…

IF IT DOESN’T RUN FASTER ON MULTIPLE CORES ON YOUR DESKTOP
…

IT’S NOT GOING TO RUN FASTER ON MULTIPLE CORES IN THE CLOUD!


CHOOSING YOUR VM SIZE
• DON’T JUST THROW BIG VMS AT EVERY PROBLEM
• SCALE OUT ARCHITECTURES HAVE NATURAL PARALLELISM
• TEST VARIOUS CONFIGURATIONS UNDER LOAD
• SOME SCENARIOS WILL BENEFIT FROM MORE CORES
Where moving data >$ parallel overhead
E.g. Video processing
• STATEFUL SERVICES
Database server requiring full network bandwidth


CACHING
• CACHING CAN IMPROVE BOTH PERFORMANCE AND SCALABILITY
Moving data closer to the consumer (Web/Worker) improves performance.
Reducing load on the hard to scale data tier


CACHING SCENARIO: WEBSITE UI IMAGES
• WEBSITE UI IMAGES • GOAL: A BETTER UI
Largely static data Serve content once
Included in every page Avoid round trip unless content changes
Minimize traffic over the wire
Fewer storage transactions
Lower load on web roles


CACHING SCENARIO: RSS FEEDS
• REGULAR RSS FEED • GOAL: A BETTER RSS FEED
Data delivered from database/storage Minimize traffic over the wire
Large content payload >1mb Fewer storage transactions
Data changes irregularly Less hits on database
Cost determined by client voracity


CACHING STRATEGIES
• CLIENT SIDE CACHING
• STATIC CONTENT GENERATION


CLIENT SIDE CACHING


CLIENT CACHING - ETAGS
• ETAG == SOFT CACHING
Header added on HTTP Response
ETag: “ABCDEFG”
Client does conditional HTTP GET
If-None-Match: “ABCDEFG”
Returns content if ETag no longer matches
Implemented natively by Windows Azure Storage
Supports client side caching
Also used for optimistic concurrency control


CLIENT CACHING - ETAGS
• BENEFITS • PROBLEMS
Prevents client downloading un- Still requires round trip to server
necessary data
May require execution of server side code
Out of the box support for simple ‘static to re-create ETag before checking
content’ scenarios.


CLIENT CACHING – CACHE-CONTROL
• Cache-Control: max-age == Hard Caching
Header added on HTTP Response
Cache-Control: max-age=2592000
Client may cache file without further request for 30 days
Client will not re-check on every request
Very useful for static files
header_logo.png
Used to determine TTL on CDN edge nodes
Set this on Blob using
x-ms-blob-cache-control


STATIC CONTENT GENERATION


• GENERATE CONTENT PERIODICALLY IN WORKER ROLE
Can spin up workers just for generation
Generate as triggered asynchronous operation
• CONTENT MAY BE
Full pages
Resources (CSS Sprites, PDF/XPS, Images etc.…)
Content fragments
• PUSH STATIC CONTENT INTO BLOB STORAGE
Serve direct out of Blob storage
May also be able to use persistent local storage


ELASTIC SCALE OUT



Compute
Compute
Inactivity
Period
Average Usage
Average Usage

Time Time

On & off workloads (e.g. batch job) Successful services needs to grow/scale
Over provisioned capacity is wasted Keeping up w/ growth is big IT challenge
Time to market can be cumbersome
Compute Cannot provision hardware fast enough

Compute
Average Usage
Average Usage

Time Time

Unexpected/unplanned peak in demand Services with micro seasonality trends
Sudden spike impacts performance Peaks due to periodic increased demand
Can’t over provision for extreme cases IT complexity and wasted capacity


DEALING WITH VARIABLE LOAD
• DEALING WITH VARIABLE LOAD TAKES TWO FORMS
• MAINTAINING EXCESS CAPACITY OR HEADROOM
Costs: paying for unused capacity
Faster availability
Asynchronous work pattern can provide buffer
• ADDING/REMOVING ADDITIONAL CAPACITY
Takes time to spin up
Requires management- human or automated
Pre-emptive or metric driven


HEAD ROOM IN WINDOWS AZURE
• WEB ROLES
Run additional web roles
Handle additional load before performance degrades

• WORKER ROLES
If possible just buffer into queues
Will be driven by tolerable level of latency
Start additional roles only if queues not clearing
Use generic workers to pool resources


RULE BASED SCALING
• USE SERVICE MANAGEMENT AND DIAGNOSTICS APIS
• ON/OFF AND PREDICTABLE BURSTING
Time based rules
• UNPREDICTABLE DEMAND AND FAST GROWTH
Monitor metrics and react accordingly


MONITOR METRICS
• PRIMARY METRICS (ACTUAL WORK DONE)
Requests per Second
Queue messages processed / interval
• SECONDARY METRICS
CPU Utilization
Queue length
Response time
• DERIVATIVE METRICS
Rate of change of queue length
Use ‘historical’ data to help predict requirements


GATHERING METRICS
• USE MICROSOFT.WINDOWSAZURE.DIAGNOSTICS.*
• CAPTURE VARIOUS METRICS VIA MANAGEMENT API
Diagnostics Infrastructure Logs
Event Logs
Performance Counters
IIS Logs
• MAY NEED TO SMOOTH/AVERAGE SOME MEASURES
• REMEMBER THE COST OF GATHERING DATA
Both performance and financial costs
Would you use Performance Counters 24/7 on a production system?
http://technet.microsoft.com/en-us/library/cc938553.aspx


EVALUATING BUSINESS RULES
• ARE REQUESTS TAKING TOO LONG?
• DO I HAVE TOO MANY JOBS IN MY QUEUE?
• HOW MUCH MONEY HAVE I SPENT THIS MONTH?

• COULD WRITE THESE INTO CODE.
• COULD BUILD SOME SORT OF RULES ENGINE.
• COULD USE THE WF RULES ENGINE.


TAKE ACTION
• ADD/REMOVE INSTANCES
Use Service Management API
• CHANGE ROLE SIZE
Requires change to *.csdef
Most suited to Worker Roles
• SEND NOTIFICATIONS
Email
IM
• MANAGE MOMENTUM
Be careful not to overshoot


SUMMARY
• DESIGNING FOR MULTIPLE INSTANCES PROVIDES
Scale out
Availability
Elasticity options
• CACHING SHOULD BE A KEY COMPONENT OF ANY WINDOWS AZURE
APPLICATION
• VARIOUS OPTIONS FOR VARIABLE LOAD
Spare capacity
Scale Out/Back
Automation possible


ANY
QUESTIONS?


THANKS,
JUAN DE ABREU
VP, Sales Director

jdeabreu@getcs.com

@juandeabreu
http://www.linkedin.com/in/juandeabreu


How to scale up, out or down in Windows Azure

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (9)

Similar a How to scale up, out or down in Windows Azure

Similar a How to scale up, out or down in Windows Azure (20)

Más de Common Sense

Más de Common Sense (7)

Último

Último (20)

How to scale up, out or down in Windows Azure