You will learn how to create file archives, upload them to Amazon S3, and manage permissions and lifetimes, giving you the ability to back up any amount of data and to retain it for as long as you'd like. A number of open source and commercial backup and archiving tools will be demonstrated, as time permits. You will also learn how to use built-in AWS facilities to quickly and easily create and restore snapshots of entire disk volumes.

1. Backup and Recovery for Linux with
Amazon S3
Learn how to back up and recover your on-premise
and cloud-based Linux systems.
Jeff Barr (@jeffbarr)
Senior Web Services Evangelist
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. What We’ll Do Today
Move quickly
Examine today’s backup challenges
Justify use of Amazon S3 for backup
Investigate backup, storage, and recovery in depth
Review options for on-premise & cloud-based backup
Give you enough information to get started today
Address your questions
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

3. Today’s Backup Challenges
Cheaper hardware
Abundant bandwidth
The paperless business
Automated data collection
Data analysis (Big Data)
Strategic importance of data
Regulatory requirements http://www.datacenterknowledge.com/archives/2012/03/20/trends-driving-data-growth-backup/
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

4. AMAZON S3 FOR BACKUP
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

5. Why Should You Back Up to Amazon S3?
Primary features:
• Offsite storage as a service
• Fully redundant
• Data stored on multiple devices in multiple facilities
• Designed for 99.999999999% durability
• Designed for 99.99% availability
• Available in all 8 AWS Regions
• Full control over data residency
• Continuous price drops
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

6. Why Should You Back Up to Amazon S3?
Additional features:
• Server-side encryption
• Transfer data through an encrypted (SSL) connection
• ACLs (Access Control Lists) to regulate access
• Object expiration
• Access logging
• Physical security
• Location control
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

7. More Reasons to Back up to S3!
Pay as you go model:
• No sticker shock
• Costs scale with usage
• No hardware to buy
Broad industry acceptance:
• 762 billion objects stored as of Q4, 2011
• Many third-party tools and applications are available
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

8. BACKUP, STORAGE, AND
RECOVERY IN DEPTH
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

9. Backup, Storage, and Recovery in Depth
There are lots of things to
thing about:
• Backup considerations
• Storage considerations
• Recovery considerations
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

10. Backup Considerations
What do you need back up?
• Files
• Databases
• Which ones (all or selected)
Full or incremental backups?
How frequently?
How much data?
Annual growth rate?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

11. More Backup Considerations
How long will a backup run take?
What about data consistency?
How long is the backup window?
The backup device:
• How much does the backup device cost?
• What is its useful life?
• Do you have a spare one?
• For each facility?
• What is the cost / GB for the backup medium?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

12. Storage Considerations
Where do you store your backups?
• Onsite – What’s your DR plan?
• Offsite – How much does that cost?
How long do you retain backups?
• Physical space limitations?
• What about media deterioration?
How quickly do you need to recover?
• How long (days) until you can retrieve your data?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

13. More Storage Considerations
How do you organize and catalog your backups?
• How do you locate the right one?
• What data is stored?
• When was it stored?
• When does it expire?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

14. Retrieval Considerations
Which backup do you need?
How long will it take to retrieve
if offsite?
What if it is missing?
What if the data is corrupt?
What is the expected retrieval /
recovery time?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

15. Expiration and Rotation Cosiderations
How long do you retain the backups?
What happens when they expire?
What is the media rotation plan?
What happens to old media?
How do you dispose of old backups?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

16. Financial Considerations
How much is your business worth?
How much is your data worth?
How much does down time cost you?
What is your medium cost / GB?
How much does your device cost?
• How many do you need?
What are the operational costs?
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

17. A TECHNICAL INTERLUDE
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

18. Some Technical Issues
Files and databases may require distinct strategies
Files
• Many objects of variable size
• Often date-stamped, allowing incremental backups
• Open files, In-flight writes
Databases
• Generally large, monolithic objects
• Open transactions
• Locks & timeouts
RAID – logical (file system) vs. physical (device) backup
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

19. Consistent Backups
Logical consistency is a must!
Operational Model
• Sync all in-memory data to disk
• Freeze database or filesystem
• fsfreeze – Linux
• mysql stop - MySQL
• Run backup
• Unfreeze
• Continue
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

20. BACKUP ARCHITECTURES
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

21. On-Premises to Cloud Backup Architectures
Transparent
Archive & Upload
Backup Tools
Backup Services
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

22. Transparent Backup / File Share
Shared file system
Local writes mirrored to the cloud
“It just works”
• Save locally
• Mirror to cloud
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

23. Transparent Backup / File Share Examples (1)
Dropbox (www.dropbox.com)
• Master copy of all files stored in S3
• Install Linux client on premises on cloud
• Backup == copy to Dropbox directory
• Recover == copy from Dropbox directory
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

24. Transparent Backup / File Share Examples (2)
S3Fuse (http://code.google.com/p/s3fs/)
• User-space file system
• Store files locally, automatically uploaded to S3
Application S3Fuse
Amazon S3 Bucket
Kernel
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

25. S3FS Configuration and Use
Configure Credentials:
Launch S3FS:
Copy files:
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

26. AWS Management Console / Shell
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

27. Archive and Upload
Use existing local tools to create archive locally
• tar, cpio, etc.
Must have sufficient local free space
Upload to S3
Manage S3 archive
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

28. Archive and Upload Example
Requirements:
• tar (standard utility)
• S3 Curl (Free download at http://aws.amazon.com/code/128)
• Supply AWS credentials:
• From command line
• In configuration file
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

29. Archive and Upload Example (1)
Create archive:
Upload to S3:
s3curl.pl --id=[…] --key=[…]
--put=backup_public_html_2012_03_28.tar
-- http://s3.amazonaws.com/jeffbarr-backup/backup_public_html_2012_03_28.tar
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

30. Archive and Upload Example (2)
File uploads to S3:
Run the tar and upload from a cron
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

31. Archive and Upload Example (3)
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

32. Archive and Upload Example (4)
Encrypt backup file!
Delete backups after
45 days!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

33. About This Simple Example…
Store as much as you want
For as long as you want
99.999999999% durability
Pay only for what you store (This file: $0.04 / month)
Encrypted
Automatic expiration
All tooling was free
Restore on premises or to cloud
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

34. Commercial Backup Tools
• Amanda Enterprise
• www.zmanda.com
• Backup agents
• Data to S3
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

35. Backup Services (1)
• www.tarsnap.com
• Local encryption
• Pay by byte/month
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

36. Backup Services (2)
FTP / SFTP / WebDAV Gateway
http://www.maluke.com/gate
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

37. CLOUD-BASED BACKUP OPTIONS
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

38. Cloud-Based Backup Tools
Everything mentioned so far still applies
• Low (internal) latency makes it even faster
• No charge to transfer between EC2 and S3
Cloud adds even more options:
• EBS snapshot backups
• RDS snapshot backups
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

39. EBS Volume Review
Create volume (1 GB to 1 TB) in an AZ
Attach to an EC2 instance
Format (mkfs) and use
Backup options:
• Traditional
• Snapshot
AFR 0.1 – 0.5%
(4% for commodity drive)
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

40. Create EBS Snapshot
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

41. Catalog EBS Snapshots
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

42. Restore EBS Volume From Snapshot
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

43. EBS Consistent Snapshot
Freeze file system
Lock databases
Take snapshot
Unlock databases
Unfreeze file system
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

44. Schedule EBS Backups
Cron and the EC2 command line tools:
ec2-create-snapshot -d "Home Volume Snapshot" vol-f12a6f9c
Third-party tools:
• Skeddly
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

45. Schedule EBS Backups - Skeddly
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

46. Things We Didn’t Cover
Amazon Relational Database Service
• Back up your RDS (MySQL or Oracle) instances just like EBS
General Database Backup
• Oracle RMAN
• MySQL Backup
AWS Storage Gateway
…Stay Tuned…
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

47. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

48. Things To Remember
The cloud makes backup and recovery easy
You can get started for pennies per month
The cloud will scale to accommodate all of your data
You retain visibility and control
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

49. For More Information
AWS Backup Site: http://aws.amazon.com/backup-storage/
Slides http://www.slideshare.net/AmazonWebServices
Webinars http://aws.amazon.com/resources/webinars
Videos http://www.youtube.com/user/AmazonWebServices
Website http://aws.amazon.com
Blog http://aws.typepad.com
White papers http://aws.amazon.com/whitepapers
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

50. Thanks For Watching!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.