1. Amazon Web Services
Introduction to Cloud Storage
Kevin Wittkopf
Enterprise Solutions Architect…

2. Cloud Storage
AWS offers multiple cloud-based storage options.
• Amazon S3 (Simple Storage Service)
• Amazon EBS (Elastic Block Storage)
• Amazon EC2 Local Instance Store (Ephemeral volumes)
• Amazon Cloudfront
Each has a unique combination of
performance, durability, cost, and interface
Often used to form a Storage Hierarchy

3. Storage by the Numbers
Cloudfront S3 EBS Local Disk
Scale Unlimited Unlimited Storage 1TB per volume 147 – 1690GB
Temporary 5TB objects ~44TB per 1-5 volumes
Storage 100 name spaces server
(buckets)
Durability N/A 99.999999999% .1-.5% AFR Ephemeral
(standard) design requires
99.99% application-level
(RRS) redundancy
Latency Low High Low Very Low
Management Zero Zero Maintenance Snapshots, Manual
Maintenance RAID, Redundancy
Consistency
Locality Global Region-Centric AZ-Specific In-machine
Interface REST or SOAP REST or SOAP Block Device Block Device
Web Services Web Services

4. S3
Amazon Simple Storage Service
©2011 Amazon Web Services May not be reused or
redistributed without permission

5. Amazon
EC2
Amazon Amazon Amazon
Amazon
EBS S3 RDS SimpleDB
©2011 Amazon Web Services May not be reused or redistributed without permission

6. Amazon Simple Storage Service
Amazon S3: Simple Storage Service
Highly scalable data storage in-the-cloud
Programmatic access via web services API
Is a Web Store, not a file system
 Optimized for WORM
 Eventually consistent
Fast, highly available and durable
Economical
©2011 Amazon Web

7. Is Your Data Storage This Resilient?
Note: Conceptual drawing only. Actual number of nodes & datacenters may vary
Region
Datacenter Datacenter
Datacenter
Node Node
1 n
...
©2011 Amazon Web

8. Amazon S3 Adoption Rate: Billions of Objects Stored
Peak Requests:
290,000+
600
per second 6 months!
500 490
400
300
262
200
102
100
40
14
2.9
0
Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q2 2011
©2011 Amazon Web Services May not be

9. Amazon S3 Namespace
Amazon S3
bucket bucket
object object object object
bucket
object object
©2011 Amazon Web Services May not be reused

10. Amazon S3 Namespace
Amazon S3
mculver-images media.mydomain.com
Beach.jpg 2009/maui/sun.jpg img1.jpg img2.jpg
public.blueorigin.com
Installer.msi img/pic1.jpg
©2011 Amazon Web Services May not be

11. Reduced Redundancy Storage Saves $$
Designed to provide 99.99% durability and 99.99%
availability of objects over a given year
This durability level corresponds to an average annual
expected loss of 0.01% of objects
Designed to sustain the loss of data in a single facility
©2011 Amazon Web

12. S3 Pricing has Three Dimensions
Storage Data Transfer Requests
Tier Norma RRS Tier Price Type Price
l
Frist 1 $0.14/G $0.093/GB All transfer in Free PUT, $0.01
TB/mo B COPY, per
POST, or 1,000
LIST
Next 49 TB $0.125/G $0.083/GB First 1 GB out Free GET and $0.01
B all other per
requests 10,000
Next 450 $0.11/GB $0.073/GB <= 10 TB out $0.12/G
TB B
Next 500 $0.095/G $0.063/GB Next 40 TB $0.09/G
TB B out B
Next 4000 $0.08/G $0.053/GB Next 100 TB $0.07/G
TB B out
©2011 Amazon Web B

13. Access Control Lists
• Control access to bucket or to object
• No inheritance from bucket to object
• Set at create, write, or any time
• List of up to 100 grants by:
– Owner
– Email
– Any AWS account
– Anyone
• Permissions:
– READ – Object or bucket
– WRITE – Create, overwrite, delete objects in
bucket
– READ_ACP – Read ACL for bucket or object
– WRITE_ACP – Overwrite ACL for bucket or object
– FULL_CONTROL – All of the above
©2011 Amazon Web Services May not be reused or redistributed without permission

14. Bucket Policy
Allow Jason or Mark to do anything in my S3 bucket
when coming from the Office internal network
{"Statement":[{
"Effect":"Allow",
"Principal":{"AWS":["4649-6425", "5243-0045"]},
"Action":"*",
"Resource":"/mybucket/*",
"Condition":{
"IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}
}}]}
©2011 Amazon Web

15. S3 Multipart Uploads
Improved throughput—upload parts in parallel
Quick recovery from any network issues—Smaller part
size minimizes the impact of restarting a failed upload due to
a network error.
Pause and resume object uploads—
 upload object parts over time
 Once you initiate a multipart upload there is no expiry
 must explicitly complete or abort the multipart upload
Begin an upload before you know the final object size—
can upload as you are create.
©2011 Amazon Web

16. Even More Amazon S3 (not so simple, is it?)
Website support
Versioning
Browser Upload to S3
Logging
Requestor Pays
Signed URLs
Bittorrent support
IAM
Meta-data
©2011 Amazon Web

17. AWS Import/Export
Get your data into AWS faster - load it onto a
portable storage device and ship it to an Amazon
data center
Faster than Internet transfer and more cost
effective than upgrading your connectivity
Use cases: data migration, offsite backup, direct
data interchange, disaster recovery
©2011 Amazon Web

18. AWS Import/Export vs. S3 upload costs
$1,200
$1,000
$800
$600 Import/Export
S3 upload
$400
$200
$0
100Gb 500Gb 1 Tb 2 Tb 3 Tb 5 Tb * 10 Tb **

19. AWS Import/Export vs. S3 upload time savings
60
50
40
Days
30 Import/Export
20 S3 upload
10
0
100 Gb 500 Gb 1 Tb 2 Tb 3 Tb 5 Tb

20. Amazon
EC2
Amazon Amazon Amazon
Amazon
EBS S3 RDS SimpleDB
©2011 Amazon Web Services May not be reused or redistributed without permission

21. EBS Features
Persistent storage
Volume lifetime is independent of any particular EC2 instance.
General purpose
Raw, unformatted, block device. Use from Linux, Solaris or
Windows.
Performance
Equal to or better than local EC2 drive.
High reliability
Built-in redundancy within availability zone.
AFR (Annual Failure Rate) between 0.1% and 1%.
Scalable
Volume sizes ranging from 1 GB to 1 TB.
Easy
Easy to create, attach, back up, restore, and delete volumes.
©2011 Amazon Web Services May not be reused or redistributed without permission

22. Amazon Web Services: Durable & Available
Note: Conceptual drawing only. The number of Availability Zones may vary
Japan
US East Region EU West Region
Availabilit Availabilit Availabilit Availabilit
y Zone A y Zone B y Zone A y Zone B Availabilit Availabilit
y Zone A y Zone B
Availabilit US West Region Availabilit Singapore
y Zone C y Zone C
Availabilit Availabilit Availabilit Availabilit
y Zone A y Zone B y Zone A y Zone B
©2011 Amazon Web Services May not be reused or redistributed without written permission

23. EBS Volumes are in a Single Availability Zone
EBS Volume 1 EBS Volume 2
Availability Zone A Availability Zone B
Note: Conceptual diagram only
©2011 Amazon Web Services May not be reused or redistributed without permission

24. How EBS Interacts With EC2 and S3
AWS Cloud
EC2 Availability Zone S3 Storage
Create Snapshot
EBS Snapshot
EBS EBS EBS EBS EBS EBS
EBS Snapshot
Clone From
Snapshot EBS Snapshot
EC2 EC2 EC2
EBS Snapshot
EBS Snapshot
Internet
©2011 Amazon Web Services May not be reused or redistributed without permission

25. What’s the Difference Between
Amazon S3 and Amazon EBS?
Amazon S3 EBS
Paradigm Object store File system
Performance Very fast Very very fast
Redundancy Across data centers Within data center
Security Public Key / Private Visible only to your
Key EC2
Pricing $0.14/GB/Mo. stored $0.10/GB/Mo. allocated
Access from the Net? Yes(1) No(2)
Typical use case Write once, read many It’s a disk drive
(1) Only with proper credentials, unless ACLs are world readable
(2) Accessible from Net if mounted to server and set up as FTP, etc.
©2011 Amazon Web Services May not be reused or redistributed without permission

26. EBS Pricing
EBS Volumes
 $0.10 per GB-month
of provisioned storage
 $0.10 per 1 million I/O requests
 No charge for mounting/unmounting
volume
EBS Snapshots to Amazon S3
 $0.14 per GB-month of data stored
 $0.01 per 1,000 PUT requests
(when saving a snapshot)
 $0.01 per 10,000 GET requests
(when loading a snapshot)
©2011 Amazon Web Services May not be reused or redistributed without permission

27. ©2011 Amazon Web Services May not be reused or
redistributed without permission
EBS Volume Lifecycle
Call CreateVolume
1 GB to 1 TB
Create
Call AttachVolume to affiliate
with one EC2 instance
Vast amounts of unused space Attach
• Format from EC2 instance
Attached
OS
&
• Mount formatted drive
Snapshot to Amazon S3 In Use
Deleted
CreateSnapshot
Call DeleteVolume Detach
Call DetachVolume

28. How Do Snapshots Work?
Time
Snapshot 2 Snapshot 3
Snapshot 1
Chunk 1
Block 1
S3
Chunk 2
Block 2
EBS Volume
Chunk 3
Block 3
Chunk 4
Block 4
©2011 Amazon Web Services May not be reused or redistributed without permission

29. Things You Should Know… (Part I)
EBS is a network resource
 Networks have variable performance
Striping will improve performance, although it’s usually fine to
start with
 4-8 drives per logical volume is observed sweet spot
 Great approach for high-performance database workloads
EBS is already redundant behind the scenes. It makes no
sense to implement RAID on top of EBS
©2011 Amazon Web Services May not be reused or redistributed without permission

30. Things You Should Know… (Part II)
Lazy loads from snapshots
 When you create a volume, space is allocated immediately
 Data from snapshot written to disk on 1st request for maximum
performance
 For maximum IO performance, run dd to touch all blocks
Like all of AWS EBS is constantly innovating.
These tips may not be appropriate in a few months
©2011 Amazon Web Services May not be reused or redistributed without permission

31. AWS Import/Export
Get your data into AWS faster - load it onto a
portable storage device and ship it to an Amazon
data center
Faster than Internet transfer and more cost
effective than upgrading your connectivity
Use cases: data migration, offsite backup, direct
data interchange, disaster recovery

32. Amazon CloudFront
Easy-to-use content delivery network
Same pay-as-you-go pricing as all of AWS
– No up-front contracts
– No long-term commitments
– Self-service sign-up
Uses Amazon S3 as the origin store
Worldwide network of edge locations
– Seattle, Palo Alto, Los Angeles, St.
Louis, Dallas, Ashburn, Newark, Miami, Amsterdam, Dub
lin, Frankfurt, London, Hong Kong, Tokyo