This document discusses using Amazon S3 and EC2 services with Ruby on Rails applications. It describes S3 as a redundant storage service that can be used for file storage and backups. EC2 is described as an on-demand computing service that provides virtual servers running Linux. The document provides examples of using S3 for asset hosting, backups, and user data storage with Rails applications. It also demonstrates how EC2 can be used to add computing capacity when needed and discusses load balancing across EC2 instances.

1. Utilizing Amazon S3 and EC2
in Rails
Jonathan Weiss, Peritor Wissensmanagement GmbH
RailsConf Europe Berlin, 2007

2. Starting Point
One machine:
• Apache
• Ruby / Rails
• MySQL
2

3. Worst Case Populariy
3

4. A Difficult Path
4

5. Problem: Backup
• High availability
• Redundancy
• Very big data sets
5

6. Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
6

7. Problem: Spontaneous Traffic
7

8. Problem: Load Fluctuation
8

9. Don‘t reinvent the wheel!
9

10. Amazon Web Services
Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S..
Redundant Data Store Virtual server per hour …….
$0,15 per GB data per month $0,10 per CPU hour
$0,10 - $0,20 per GB traffic $0,10 - $0,20 per GB traffic
10

11. S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
11

12. S3 - Buckets
S3
• Unique over all S3
• Contains many
MyBucket_name www.peritor.com
key-value-metadata tupel
foto_5.jpg site/screen.css
• Cannot contain other buckets!
backups/januar.zip site/script.js
• Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso
12

13. S3 with AWS::S3
Upload
Download
13

14. EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• 1,7 GHz
• 1,75 GB RAM
• 160 GB local file system (not persistent!)
• Use your favorite Linux distro (Linux 2.6.16),
Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
14

15. EC2 Tools
List availabe images
Start a new instance
Login with SSH
Shutdown instance
15

16. amazon-ec2 gem
Setup
Usage
16

17. And now?
How does this solve
my problems?
17

18. S3 - Backup
• s3sync.rb • S3Browser
• Brackup • Firefox S3 Organizer extension
• Jungle Disk •…
• S3 FUSE
• s3DAV
• Duplicity
18

19. s3sync.rb
Backup
Restore
19

20. S3 Asset Host
20

21. S3 Asset Host
Setup DNS
Rails configuration
21

22. S3 Asset Host
welcome.rhtml template
Output
22

23. S3 - Authenticated User Data
23

24. attachment_fu Rails plugin
Setup
24

25. attachment_fu Rails plugin
Upload
Download
25

26. On-Demand Computing with EC2
Time based, e.g. with cron
26

27. On-Demand Computing with EC2
Load based, e.g. with Monit
27

28. On-Demand Computing with EC2
28

29. EC2 for extra capacity
29

30. Load Balancer / Proxy
Example mod_proxy_blancer:
• Talks to multiple backend servers
(Mongrel)
• Central Proxy/Load-Balancer
configuration that has knowledge
about nodes
• Typically proxy restart on config
change
30

31. Swiftiply
Swiftiply Proxy:
• Multiple backend clients have a
persistent connection to the backend
port
• Proxy listens on cluster port for
requests and forwards them
No re-configuration
31

32. Swiftiply Proxy
Installation Configuration
Start
32

33. Swiftiplied Mongrel
• Gem plugin that patches Mongrel
• Transforms Mongrel in Swiftiply client
• Experimental
Start
or
33

34. EC2 on Demand before Swiftiply
34

35. EC2 on Demand with Swiftiply
35

36. EC2 on Demand with Swiftiply
36

37. Ressources
• Amazon Web Services
http://aws.amazon.com
• Swiftiply
http://swiftiply.swiftcore.org
• Attachment_fu
http://svn.techno-weenie.net/projects/plugins/attachment_fu/
• Elastic Rails
http://www.elasticrails.com
• Capazon
http://capazon.rubyforge.org
• RightScale
http://www.rightscale.com
37

38. Peritor Wissensmanagement GmbH
Lenbachstraße 2
12157 Berlin
Internet: www.peritor.com Telefon: +49 (0)30 69 40 11 94
Email: info@peritor.com Telefax: +49 (0)30 69 40 11 95
© Peritor Wissensmanagement GmbH - All Rights Reserved
38