AWS 101 Event - 16 July 2013

AWS 101
Ianni Vamvadelis
Manager, Solution Architecture

Agenda
10:00 - 10:45 Presentation – AWS 101
Introducing the concepts behind AWS such as utility computing and elasticity.
10:45 - 11:00 Coffee Break
11:00 - 11:45 Presentation and Demonstration
Live demonstration and interactive walkthrough

What we are going to cover
Keypairs
Security groups
EC2 instances
Metadata service
Autoscaling
Amazon Machine Images
S3
CloudFront
Elastic Load balancer
RDS
Feel free to follow along on your laptops

Consumer
Business
Tens of millions of
active customer
accounts
Eight countries:
US, UK, Germany,
Japan, France,
Canada, China, Italy
Seller
Business
Sell on Amazon
websites
Use Amazon
technology for your
own retail website
Leverage Amazon’s
massive fulfillment
center network
IT Infrastructure
Business
Cloud computing
infrastructure for
hosting web-scale
solutions
Hundreds of
thousands of
registered
customers in over
190 countries

Deep experience in
building and
operating global web
scale systems
About Amazon
Web Services
?
…get into cloud computing?
How did Amazon…

Over 10 years in the making
Enablement of sellers on Amazon
Internal need for scalable deployment environment
Early forays proved developers were hungry for more

AWS Mission
Enable businesses and
developers to use web
services* to build scalable,
sophisticated applications.
*What people now call “the cloud”

Each day AWS adds the equivalent server
capacity to power Amazon when it was a
global, $5B enterprise

Each day AWS adds the equivalent server
capacity to power Amazon when it was a
global, $5B enterprise
$5.2B retail business
7,800 employees
A whole lot of servers
2003

Total Number of Objects Stored in Amazon S3
Objects in S3
2.9 Billion 14 Billion 40 Billion
102 Billion
262 Billion
762 Billion
1.7 Trillion
2 Trillion
Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q4 2011 Q4 2012 Q2 2013
Over 1.1 Million
requests per second

EMR Jobs
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
3,500,000
4,000,000
3.7 M clusters
launched since May 2010

On demand Pay as you go
Uniform Available
Utility computing

Utility computing
Uniform Available

Utility computing
Compute
Storage
Security
Scaling
Database
Networking
Monitoring
Messaging
Workflow
DNS
Load Balancing
BackupCDN
Uniform Available

On a global footprint
Region
US-WEST (N. California) EU-WEST (Ireland)
ASIA PAC
(Tokyo)
ASIA PAC
(Singapore)
US-WEST (Oregon)
SOUTH AMERICA (Sao
Paulo)
US-EAST (Virginia)
GOV CLOUD
ASIA PAC
(Sydney)

Availability Zone

Edge Locations
Dallas(2)
St.Louis
Miami
JacksonvilleLos Angeles (2)
Palo Alto
Seattle
Ashburn(2)
Newark
New York (3)
Dublin
London(2)
Amsterdam (2)
Stockholm
Frankfurt(2)
Paris(2)
Singapore(2)
Hong Kong
Tokyo (2)
Sao Paulo
South Bend
San Jose
Osaka
Milan
Sydney
Madrid

At the end of a web service
ec2-run-instances ami-b232d0db
--instance-count 3
--availability-zone eu-west-1a
--instance-type m1.small
--instance-count 5
--availability-zone eu-west-1c
--instance-type m1.medium

--instance-count 2
--availability-zone us-east-1b
--instance-type m1.xlarge
--instance-count 2
--availability-zone us-east-1d
--instance-type m1.xlarge

elb-create-lb
myLoadBalancer
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1c
--min-size 2
--max-size 200
ec2-authorize default -p 80

Traditional IT
capacity
Elastic capacity
Capacity
Time
Your IT needs

On and Off Fast Growth
Variable peaks Predictable peaks
Elastic capacity

Elastic capacity
On and Off Fast Growth
Predictable peaksVariable peaks
WASTE
CUSTOMER DISSATISFACTION

Elastic cloud capacity
Traditional
IT capacity
Your IT needs
Time
Capacity
Elastic capacity

Elastic capacity
Fast GrowthOn and Off
Predictable peaksVariable peaks

503
Service Temporarily Unavailable
The server is temporarily unable
to service your request due to
maintenance downtime or capacity
problems. Please try again later.

Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Typical weekly traffic to
Amazon.com

November traffic to Amazon.com
November

Provisioned capacity
November

76%
24%
Provisioned capacity
November

November 10th 2010
Turned off last physical web server of
Amazon.com

November 10th 2010
Turned off last physical web server of
Amazon.com
October 31st 2011
Turned off last web servers supporting
European business

Time: +00h
<10 cores
Elastic Capacity

Time: +24h
>1500
cores
Elastic Capacity

Time: +72h
<10 cores
Elastic Capacity

Time: +120h
>600 cores
Elastic Capacity

NumberofEC2Instances
4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/20084/17/20084/13/2008
40 servers to 5000 in 3 days
EC2 scaled to peak
of 5000 instances
“Techcrunched”
Launch of Facebook
modification
Steady state of ~40
instances

Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Reference Model
security

Compute Storage
Database
App Services
Networking
Global infrastructure
Regions
An independent collection of AWS resources in a
defined geography
A solid foundation for meeting location-dependent
privacy and compliance requirements

Compute Storage
Database
App Services
Networking
Availability Zones
Designed as independent failure zones
Physically separated within a typical metropolitan
region

Compute Storage
Database
App Services
Networking
Edge Locations
To deliver content to end users with lower latency
A global network of edge locations
Supports global DNS infrastructure (Route53) and
Cloud Front CDN

Compute Storage
Database
App Services
Networking
Networking
Direct Connect
Dedicated connection to AWS
VPN Connection
Secure internet connection to AWS
Virtual Private Cloud
Private, isolated section of the AWS Cloud
Route 53
Highly available and scalable Domain Name
Service

Compute Storage
Database
App Services
Networking
Compute
Vertical
Scaling
From $0.02/hr
Elastic Compute Cloud (EC2)
Basic unit of compute capacity
Range of CPU, memory & local disk options
13 Instance types available, from micro to cluster
compute
Feature Details
Flexible Run windows or linux distributions
Scalable Wide range of instance types from micro to cluster
compute
Machine Images Configurations can be saved as machine images
(AMIs) from which new instances can be created
Full control Full root or administrator rights
Secure Full firewall control via Security Groups
Monitoring Publishes metrics to Cloud Watch
Inexpensive On-demand, Reserved and Spot instance types
VM Import/Export Import and export VM images to transfer
configurations in and out of EC2

Compute Storage
Database
App Services
Networking
Compute
Auto-scaling
Automatic provisioning of compute resources based
upon demand, configuration or schedule
Trigger auto-
scaling policy
Feature Details
Control Define minimum and maximum instance pool sizes
and when scaling and cool down occurs
Integrated to
CloudWatch
Use metrics gathered by CloudWatch to drive scaling
Instance types Run auto scaling for on-demand instances and spot.
Compatible with VPC
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 4
--max-size 200

Compute Storage
Database
App Services
Networking
Compute
Elastic Load Balancing
Create highly scalable applications
Distribute load across EC2 instances in multiple
availability zones
Feature Details
Auto-scaling Automatically scales to handle request volume
Available Load balance across instances in multiple availability
zones
Health checks Automatically checks health of instances and takes
them in or out of service
Session stickiness Route requests to the same instance
Secure sockets layer Supports SSL offload from web and application
servers with flexible cipher support
Monitoring Publishes metrics to Cloud Watch

Compute Storage
Database
App Services
Networking
Storage
S3 - Durable storage, any
object
99.999999999% durability of objects
Unlimited storage of objects of any type
Up to 5TB size per objectFeature Details
Flexible object store Buckets act like drives, folder structures within
Access control Granular control over object permissions
Server-side encryption 256bit AES encryption of objects
Multi-part uploads Improved throughput & control
Object versioning Archive old objects and version new ones
Object expiry Automatically remove old objects
Access logging Full audit log of bucket/object actions
Web content hosting Serve content as web site with built in page handling
Notifications Receive notifications on key events
Import/Export Physical device import/export service

Compute Storage
Database
App Services
Networking
Storage
Elastic Block Store
High performance block storage device
1GB to 1TB in size
Mount as drives to instances
Feature Details
High performance file
system
Mount EBS as drives and format as required
Flexible size Volumes from 1GB to 1TB in size
Secure Private to your instances
Available Replicated within an Availability Zone
Backups Volumes can be snapshotted for point in time restore
Monitoring Detailed metrics captured via Cloud Watch

Compute Storage
Database
App Services
Networking
Database
Relational Database Service
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations
Feature Details
Platform support Create MySQL, SQL Server and Oracle RDBMS
Preconfigured Get started instantly with sensible default settings
Automated patching Keep your database platform up to date automatically
Backups Automatic backups and point in time recovery and full
DB backups
Backups Volumes can be snapshotted for point in time restore
Failover Automated failover to slave hosts in event of a failure
Replication Easily create read-replicas of your data and
seamlessly replicate data across availability zones

Compute Storage
Database
App Services
Networking
Database
Amazon Relational Database Service
(Amazon RDS) databases stores forum
threads, site content, and project
configuration data.
High availability Multi-AZ database
deployment to handle live game metadata
and user-generated content.
Enterprise-grade fault tolerance for
protecting customer data.
By managing time-consuming database
administration tasks, Amazon RDS allows
SEGA to focus on business critical
applications.

Compute Storage
Database
App Services
Networking
Database
DynamoDB
Provisioned throughput NoSQL database
Fast, predictable performance
Fully distributed, fault tolerant architecture
Feature Details
Provisioned throughput Dial up or down provisioned read/write capacity
Predictable
performance
Average single digit millisecond latencies from SSD
backed infrastructure
Strong consistency Be sure you are reading the most up to date values
Fault tolerant Data replicated across availability zones
Monitoring Integrated to Cloud Watch
Secure Integrates with AWS Identity and Access
Management (IAM)
Elastic MapReduce Integrates with Elastic MapReduce for complex
analytics on large datasets

Compute Storage
Database
App Services
Networking
Application Services
Amazon SQS
Processing
task/processing trig
Processing results
Amazon SQS
Reliable, highly scalable, queue
service for storing messages as they
travel between instances
Feature Details
Reliable Messages stored redundantly across multiple
availability zones
Simple Simple APIs to send and receive messages
Scalable Unlimited number of messages
Secure Authentication of queues to ensure controlled access

Task A
Task B
(Auto-scaling)
Task C
2
3
1
Compute Storage
Database
App Services
Networking
Feature Details
Process state Maintain application state across complex workflows
in a reliable and available manner
Tracking Tracks executions and log process for audit purposes
Consistency Ensures processing tasks are executed and duplicity of
events does not occur
Simple Simple Decider and Task programming model for
rapid integration
Simple Workflow
Reliably coordinate processing steps
across applications
Integrate AWS and non-AWS resources
Manage distributed state in complex
systems

Compute Storage
Database
App Services
Networking
Cloud Search
Elastic search engine based upon
Amazon A9 search engine
Fully managed service with
sophisticated feature set
Scales automatically
Document
Server
Results
Search
Server
Feature Details
Auto-scaling Automatically scales based upon request volumes and
data volumes
High performance In memory operation means consistently low latency
for search results
Sophisticated features Support for faceting, stemming, synonyms, stop
words and custom rank expressions
Low cost Elastic service, pay for what you use

Compute Storage
Database
App Services
Networking
CloudFront
World-wide content distribution
network
Easily distribute content to end users
with low latency, high data transfer
speeds, and no commitments.
Feature Details
Fast Multiple world-wide edge locations to serve content
as close to your users as possible
Integrated with other
services
Works seamlessly with S3 and EC2 origin servers
Dynamic content Supports static and dynamic content from origin
servers
Streaming Supports rtmp from S3 and includes support for live
streaming from Adobe FMS and Microsoft Media
Server
London
Paris
NY
Served from S3
/images/*
3
Served from EC2
*.php
2
Single CNAME
www.mysite.com
1

Compute Storage
Database
App Services
Networking
Deployment & Admin
Elastic Beanstalk
One-click deployment from Eclipse, Visual Studio and
Git
Rapid deployment of applications
All AWS resources automatically created
Feature Details
Platform support Containers for Java, .net and PHP
Resource creation Creates load balancer, instances, autoscaling and
monitoring automatically
Monitoring & Logs Integrated with Cloud Watch and consolidates server
logs
Versioning Manage versions of applications and easily rollback
deployments
Notifications Receive alerts on key events
Full resource access Access all underlying AWS resources as necessary

Compute Storage
Database
App Services
Networking
Deployment & Admin
OpsWorks
DevOps focused managed application stacks
Underlying Chef recipes allow for complete
customisation
Feature Details
Platform support Chef recipes allows for community expansion for
platform components such as Solr, NgniX etc
Resource creation Customizable deployments, rollback, partial
deployments, patch management, automatic
instance scaling, and auto healing
Layered Manage logical application layers and combine
into stacks.

Compute Storage
Database
App Services
Networking
Deployment & Admin
Cloud Formation
Automate creation of ‘stacks’ in a repeatable way
Scripting framework for AWS resource creation
Feature Details
Platform support Support for AWS resources from EC2 to IAM
Resource creation Creates AWS resources behind the scenes and reports
on progress
Declarative Specify stacks in JSON format and source control your
environments
Customizable Drive stack creation with paramaters

Compute Storage
Database
App Services
Networking
Deployment & Admin
Identity & Access Management
Granular control of user rights with AWS
Automated granting of EC2 service rights
Software Developer Kits
Comprehensive support of programming models for
using AWS services

+ others
Simple Email Service
Simple Notification Service
ElastiCache
Elastic MapReduce
CloudWatch
…

Introducing Amazon
Redshift
Data Warehousing the AWS Way
Easily and rapidly analyze
petabytes of data
1/10 the cost of traditional data
warehouses
Automated deployment &
administration
Compatible with popular BI tools

Orchestration service for data-driven workflows
Create automated and scheduled data flows
Pre-integrated with AWS data sources
Easily connect with 3rd party & on-premises sources
AWSData Pipeline
Your Analytics

Amazon S3
Input Bucket
Amazon S3
Input Bucket
Amazon S3
Output Bucket
Amazon S3
Output Bucket
Amazon S3
Output Bucket
Transcoding Pipeline 1
Kindle Fire HD720p Custom



Inside a Transcoding Pipeline
Transcoding Job n Transcoding Job 2 Transcoding Job 1
Amazon Elastic Transcoder

Foundation Services
Compute Storage Database Networking
AWS Global
Infrastructure Regions
Availability Zones
Edge Locations
Amazon
Shared responsibility

Foundation Services
Compute Storage Database Networking
AWS Global
Infrastructure Regions
Availability Zones
Edge Locations
Client-side Data Encryption & Data
Integrity Authentication
Server-side Encryption
(File System and/or Data)
Network Traffic Protection
(Encryption/Integrity/Identity)
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Customer Data
Amazon
Shared responsibility
You

Certifications
SOC 1 Type 2 (formerly
SAS-70)
ISO 27001
PCI DSS for EC2, S3,
EBS, VPC, RDS, ELB,
IAM
FISMA Moderate
Compliant Controls
HIPAA & ITAR
Compliant Architecture
Physical Security
Datacenters in
nondescript facilities
Physical access strictly
controlled
Must pass two-factor
authentication at least
twice for floor access
Physical access logged
and audited
HW, SW, Network
Systematic change
management
Phased updates
deployment
Safe storage
decommission
Automated monitoring
and self-audit
Advanced network
protection
Security standards
http://aws.amazon.com/security

So what are
we going to do
today?

Availability Zone Region
Instance

Instance S3

Instance S3
Cloud
Front

Instance S3
Cloud
Front
RDS

Instance Instance
Elastic Load
Balancer
Cloud
Front
S3
RDS

Instance Instance
Elastic Load
Balancer
Cloud
Front
S3
Auto
scaling
Group
RDS

Enjoy some coffee /
tea,
come back at 11:00,
and we’ll build it

Ask questions
(it will fill time when we wait for
things to launch)
There will be a recording of this
demo on YouTube, so don’t
worry if you miss anything

bootstrapping
passing data to an instance

Instance
request
User
data
Meta-data
service

Instance
request
User
data
Instance
Meta-data
service

Script executed on launch:
<script>
ipconfig /all > c:ipconfig.txt
netstat > c:netstat.txt
</script>

Script executed on launch:
#!/bin/sh
yum -y install httpd php php-mysql
chkconfig httpd on
/etc/init.d/httpd start

Public Key
Inserted by Amazon into
each EC2 instance that
you launch
Private Key
Downloaded and stored
by you
EC2
Instance
Comms secured
with private key

security groups
instance firewalling

Security Group
instance
Port 80
(HTTP)
Port 22
(SSH)
Name
Description
Protocol
Port range
IP Address, range,
or another security
group

index.php
Reads instance meta-data

Some php code that gets the data
<?php
// get the instance id
$url = "http://169.254.169.254/latest/meta-data/instance-id";
$instance_id = file_get_contents($url);
// get the AZ where the instance is running
$url = "http://169.254.169.254/latest/meta-data/placement/availability-zone";
$zone = file_get_contents($url);
// get the security group it is in
$url = "http://169.254.169.254/latest/meta-data/security-groups";
$group = file_get_contents($url);
// get the public DNS name
$url = "http://169.254.169.254/latest/meta-data/public-hostname";
$hostname = file_get_contents($url);
?>

And diplays it
Instance ID: <?php echo $instance_id; ?>
Availability Zone: <?php echo $zone; ?>
Security Group: <?php echo $group; ?>

Availability Zone Availability Zone
Region
Availability Zone
Instance Instance Instance Instance Instance Instance
Elastic Load Balancer

auto-scaling
elastic server pool

Describes what Auto
Scaling will create when
adding
Instances
AMI
Instance Type
Security Group
Instance Key Pair
Only one active launch
configuration at a time
Auto Scaling will terminate
instances with old launch
configuration first
rolling update
Auto Scaling managed
grouping of EC2
instances
Automatic health check to
maintain pool size
Automatically scale the
number of instances by
policy – Min, Max, Desired
Automatic Integration with
ELB
Automatic distribution &
balancing across AZs
Parameters for
performing an Auto
Scaling action
Scale Up/Down and by
how much
ChangeInCapacity (+/- #)
ExactCapacity (#)
ChangeInPercent (+/- %)
Cool Down (seconds)
Policy can be triggered by
CloudWatch events
Launch Configuration Auto-Scaling Group Auto-Scaling Policy

as-create-launch-config
--image-id ami-50348c51
--instance-type t1.micro
--key apac-tok
--group 'Web Servers’
--launch-config 101-launch-config
Create a launch configuration:

as-create-auto-scaling-group 101-as-group
--availability-zones ap-northeast-1a
--launch-configuration 101-launch-config
--load-balancers myELB
--max-size 5
--min-size 1
Create an auto-scaling group:

as-put-scaling-policy 101ScaleUpPolicy
--auto-scaling-group 101-as-group
--adjustment=1
--type ChangeInCapacity
--cooldown 300
Create an auto-scaling policy (scale up):

as-put-scaling-policy 101ScaleDownPolicy
--auto-scaling-group 101-as-group
"--adjustment=-1"
--type ChangeInCapacity
--cooldown 300
Create an auto-scaling policy (scale down):

To do this you will need:
An AWS Account
Auto-scaling tools installed
Find out more:
aws.amazon.com

AWS 101 Event - 16 July 2013

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a AWS 101 Event - 16 July 2013

Similar a AWS 101 Event - 16 July 2013 (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

Último

Último (20)

AWS 101 Event - 16 July 2013