SlideShare una empresa de Scribd logo
1 de 57
tweet #reinvent
•
•
•
•
•
•
SIMPLE STORAGE SERVICE
DURABILITY
UNIQUE CUSTOMER
    OBJECTS
PEAK TRANSACTIONS
   PER SECOND
99.99% vs. 99.999999999%
durability of objects over a
given year
Customer Decides Where Applications and Data Reside
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
• Disaster Recovery
                                          • Backup                                   Region 2
                               Region 1
                                          • Distribution




• Archive
• Load for Elastic MapReduce
• Migrate / Sync

                                                   www.attunitycloudbeam.com

                          Enterprise               • SaaS-based solution
                                                   • Simplicity, “click and replicate”
                                                   • Speed, optimized and elastic transfer
                                                   • Safe, secure and guaranteed delivery
• LOW-COST ARCHIVING SERVICE
PER GB / MONTH
PER TB / YEAR
DURABILITY
DATA RETRIEVAL
LOW-COST ARCHIVING SERVICE
Archive after 30 days


   logs/file1
   logs/file2
   logs/file3   logs




                                               Amazon Glacier
My S3 bucket
Expire after
                 365 days



   logs/file1
   logs/file2
                        logs
   logs/file3




                          Amazon Glacier
My S3 bucket
Storage Options


                        Amazon                     Reduced
                 Data
                        Glacier                  Redundancy
                                                   Storage
                                  Standard Storage
Amazon Glacier
                                   Amazon S3
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
Glacier Transforms tiered storage
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
HSM
with AWS                                                     Amazon               Amazon
                          SAN
                                                               S3     AWS Cloud
                                                                                  Glacier
                    Corporate Data Center

versus


Traditional
Approach
to HSM
                                                                         offsite tape
              SAN                 tier 2             disk
                                                                           storage
                                 storage            backup
                            Corporate Data Center
Compliance
with AWS      O/S image               Amazon           Amazon
                                        S3 AWS Cloud   Glacier

versus


Traditional
Approach
                                                   off-site tape
              O/S image          disk
                                                     storage
                                backup
                          Corporate Data Center
Options to consider
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
5
                                   HTTP –
               2   fasp                                    3
                                   multipart




                                                       4          Parallel Transcoding
        fasp       5                                               14 instances: 3 min

                                                                             Herndon, VA
1   6

                   1. Video broadcast capture
                   2. High-speed upload Direct-to-S3
                   3. Scale out parallel transcode
`
                   4. Deliver back to S3
                   5. High-speed download from S3 to UFC
                   6. Insert into CMS for streaming to mobile devices
Common mistakes
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
What is Imgur?
• A simple image sharer
• Has the most viral images on the
  Internet
• Anyone can upload as many
  images as they want – without an
  account
• 2,000,000 images uploaded per
  day
   • That’s 23 images per second
• Can be embedded and shared on
  any site
The greatest image site. Full of the all the wonders
and magic of the interwebs. Be forewarned, time has
been known to quicken in this realm.

“I spent half a day on Imgur, and it was the greatest 6 hours
of my life.”


                                           - Urban Dictionary
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
• Started as a side project while at
    Ohio University
•   Redditors needed a place to host
    their images
•   Organically grew into a business
•   Alan was the only developer for 3
    years
•   Moved to San Francisco
•   Now a team of 7
    •   (600 million pageviews per engineer)
•   Every month. There are:                  • 11 minutes average visit duration
    • 2.9 billion page views                 • 11 pages per visit
    • 38 billion image views (images loaded)
                                             • 46th biggest site in the US
    •   54 million unique visitors             –   (according to alexa.com)
    •   4.7 petabytes of bandwidth used
                                           • 600 million objects stored in S3
    •   62 million images uploaded
                                                                              * All data as of Nov
                                                                                             2012
• Pageviews are growing 15% every
  month.
• How are we able to support this kind of
  growth?
User make a request for an
image


                         (Don’t do this!)
User make a request for an image
User make a request for an image
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
• Site traffic is increasing more than ever. How many more
  servers do we need?
• Hardware failures
• Tweaking every little thing is really hard and easy to get
  wrong, but necessary
• There’s only one man doing all this; how can we make his
  life easier, while scaling the site at the same time?
•   Autoscaling is awesome
•   Automated DB backups are awesome
•   Security features are awesome
•   Much easier to manage in the long run
•   Because everything’s managed, you
    require less admins to look over
    everything all the time
•   AWS has managed solutions for all the
    core services your website needs
    (server, database, cache, backups,
    security, etc.)
• Lots of new stuff to learn and
  set up
• Possible downtime during
  migration
• Very time consuming at first
  because you’re reconfiguring
  your entire stack
•   AWS has a lot of services; find out which
    ones can work for you and how

•   Use the price calculator:
    http://calculator.s3.amazonaws.com/calc5.html

•   Read the docs: http://aws.amazon.com

•   Set up a test environment

•   Install the AWS SDK

•   Call AWS if you have questions.
    •   (You don’t need AWS Support to call in)

•   Start coding!
•   How do you get all your data to S3?
    •   Duplicate writes: 1 to native, 1 to S3.
    •   Upload all your data to S3 in parallel.We had 12
        background processes running around the
        clock all uploading a different subset of data to
        S3 – it took 2 weeks to finish.
    •   No need to store more than one copy
    •   Turn on versioning for even more protection

•   Very similar process for Amazon RDS
• There’s no web interface
• Have to do everything from
    command line
•   Confusing terminology
•   Hard to verify that it’s working as
    intended
•   But in the end, it’s amazing
•   If you’re not using it, you’d better
    have a really good reason why
•   EC2:
    •   Maximum performance with RAID0 Elastic Block Store
    •   RAID0 EBS requires a pretty significant amount of maintenance overhead
    •   Have to come up with your own backup plan

•   RDS:
    •   Will provide very good performance out of the box (but not maximum)
    •   Management console is fantastic
    •   Easy to upgrade instances
    •   High availability and read-only slaves are a click away
    •   Managed service, which makes it more expensive

•   If you enjoy tuning every last little bit for maximum performance, then you
    can consider EC2 + EBS RAID 0

•   Still on the fence? Go with RDS
• There’s no access to the underlying file system

• Migrating requires a dump and an import of your data, which
  is extremely time consuming for large databases

• No access to the logs when things break

• We were able to do it live – without taking the site down – but
  with lots of headaches
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012
•   Wed (1:00 p.m.–1:50 p.m.) MED203: Scalable Media Processing with AWS
•   Wed (2:05 p.m.–2:55 p.m.) MED202: Netflix’s Transcoding Transformation
•   Wed (3:25 p.m.–4:15 p.m.) MED303: Addressing Security in Media Workflow


•   Thu (10:30 a.m.–11:20 a.m.) STG205: Amazon S3: Reduce costs, save time, and better
    protect your data
•   Thu (11:35 a.m.–12:25 p.m.) STG203: Cloud Storage War Stories: From the front lines of
    some of the biggest battles
•   Thu (4:05 p.m.–4:55 p.m.) STG302: Archive in the Cloud with Amazon Glacier


    http://aws.amazon.com/s3/
    http://aws.amazon.com/glacier/faqs/
    http://aws.amazon.com/digital-media/


•   Wed (1:00 p.m.–1:50 p.m.) STG201: Understanding AWS Storage Options
We are sincerely eager to      tweet #reinvent
 hear your feedback on this
presentation and on re:Invent.

 Please fill out an evaluation
   form when you have a
            chance.

Más contenido relacionado

La actualidad más candente

Scaling the Platform for Your Startup
Scaling the Platform for Your StartupScaling the Platform for Your Startup
Scaling the Platform for Your StartupAmazon Web Services
 
Windows Azure Design Patterns
Windows Azure Design PatternsWindows Azure Design Patterns
Windows Azure Design PatternsDavid Pallmann
 
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...Amazon Web Services
 
Scaling on AWS for the First 10 Million Users
Scaling on AWS for the First 10 Million Users Scaling on AWS for the First 10 Million Users
Scaling on AWS for the First 10 Million Users Amazon Web Services
 
SDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerSDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerKorea Sdec
 
Getting started with Amazon ElastiCache
Getting started with Amazon ElastiCacheGetting started with Amazon ElastiCache
Getting started with Amazon ElastiCacheAmazon Web Services
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Adrian Cockcroft
 
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)Amazon Web Services
 
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍Amazon Web Services Korea
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connectAdrian Cockcroft
 
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Amazon Web Services
 
Introduction to AWS Database Services
Introduction to AWS Database ServicesIntroduction to AWS Database Services
Introduction to AWS Database ServicesAmazon Web Services
 
Deep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduceDeep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduceAmazon Web Services
 
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...Amazon Web Services
 
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...Rustem Feyzkhanov
 
Deep Dive: Scaling Up to Your First 10 Million Users
Deep Dive: Scaling Up to Your First 10 Million UsersDeep Dive: Scaling Up to Your First 10 Million Users
Deep Dive: Scaling Up to Your First 10 Million UsersAmazon Web Services
 
Amazon CloudFront - Delivering Dynamic Content From The Edge
Amazon CloudFront - Delivering Dynamic Content From The EdgeAmazon CloudFront - Delivering Dynamic Content From The Edge
Amazon CloudFront - Delivering Dynamic Content From The EdgeAmazon Web Services
 

La actualidad más candente (20)

Scaling the Platform for Your Startup
Scaling the Platform for Your StartupScaling the Platform for Your Startup
Scaling the Platform for Your Startup
 
Amazon rds
Amazon rdsAmazon rds
Amazon rds
 
Windows Azure Design Patterns
Windows Azure Design PatternsWindows Azure Design Patterns
Windows Azure Design Patterns
 
Startup Showcase - Mojang
Startup Showcase - MojangStartup Showcase - Mojang
Startup Showcase - Mojang
 
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...
Amazon RDS & Amazon Aurora: Relational Databases on AWS - SRV206 - Atlanta AW...
 
Scaling on AWS for the First 10 Million Users
Scaling on AWS for the First 10 Million Users Scaling on AWS for the First 10 Million Users
Scaling on AWS for the First 10 Million Users
 
SDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerSDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuer
 
Deep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDBDeep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDB
 
Getting started with Amazon ElastiCache
Getting started with Amazon ElastiCacheGetting started with Amazon ElastiCache
Getting started with Amazon ElastiCache
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
 
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)
AWS Summit London 2014 | Scaling on AWS for the First 10 Million Users (200)
 
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
 
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
 
Introduction to AWS Database Services
Introduction to AWS Database ServicesIntroduction to AWS Database Services
Introduction to AWS Database Services
 
Deep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduceDeep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduce
 
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...
 
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...
DataTalks.Club - Building Scalable End-to-End Deep Learning Pipelines in the ...
 
Deep Dive: Scaling Up to Your First 10 Million Users
Deep Dive: Scaling Up to Your First 10 Million UsersDeep Dive: Scaling Up to Your First 10 Million Users
Deep Dive: Scaling Up to Your First 10 Million Users
 
Amazon CloudFront - Delivering Dynamic Content From The Edge
Amazon CloudFront - Delivering Dynamic Content From The EdgeAmazon CloudFront - Delivering Dynamic Content From The Edge
Amazon CloudFront - Delivering Dynamic Content From The Edge
 

Destacado

Three Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active ArchiveThree Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active ArchiveAvere Systems
 
Digital Media Ingest and Storage Options on AWS
Digital Media Ingest and Storage Options on AWSDigital Media Ingest and Storage Options on AWS
Digital Media Ingest and Storage Options on AWSAmazon Web Services
 
SUN TV NETWORK LIMITED
SUN TV NETWORK LIMITEDSUN TV NETWORK LIMITED
SUN TV NETWORK LIMITEDARVIND D
 
ximena araneda - The Next Generation MAM Systems
ximena araneda - The Next Generation MAM Systemsximena araneda - The Next Generation MAM Systems
ximena araneda - The Next Generation MAM SystemsFIAT/IFTA
 
AWS Customer Presentation - Smugmug
AWS Customer Presentation - SmugmugAWS Customer Presentation - Smugmug
AWS Customer Presentation - SmugmugAmazon Web Services
 
ARC304 Solutions in Action - AWS re: Invent 2012
ARC304 Solutions in Action - AWS re: Invent 2012ARC304 Solutions in Action - AWS re: Invent 2012
ARC304 Solutions in Action - AWS re: Invent 2012Amazon Web Services
 
AWS Customer Presentation - Turbo10
AWS Customer Presentation - Turbo10AWS Customer Presentation - Turbo10
AWS Customer Presentation - Turbo10Amazon Web Services
 
Choosing the Right Data Storage - Carlos Conde - AWS Summit Paris
Choosing the Right Data Storage - Carlos Conde - AWS Summit ParisChoosing the Right Data Storage - Carlos Conde - AWS Summit Paris
Choosing the Right Data Storage - Carlos Conde - AWS Summit ParisAmazon Web Services
 
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012Amazon Web Services
 
AWS Update | London - Amazon Glacier
AWS Update | London - Amazon GlacierAWS Update | London - Amazon Glacier
AWS Update | London - Amazon GlacierAmazon Web Services
 
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012Amazon Web Services
 
AWS Customer Presentation - MessageSling
AWS Customer Presentation - MessageSlingAWS Customer Presentation - MessageSling
AWS Customer Presentation - MessageSlingAmazon Web Services
 
AWS Customer Presentation - Nutsie
AWS Customer Presentation -  Nutsie AWS Customer Presentation -  Nutsie
AWS Customer Presentation - Nutsie Amazon Web Services
 
AWS Customer Presentation - Mahalo
AWS Customer Presentation - Mahalo AWS Customer Presentation - Mahalo
AWS Customer Presentation - Mahalo Amazon Web Services
 
AWS Customer Presentation - HotPads
AWS Customer Presentation - HotPadsAWS Customer Presentation - HotPads
AWS Customer Presentation - HotPadsAmazon Web Services
 
Scalability and Availability for Marketing Campaigns
Scalability and Availability for Marketing CampaignsScalability and Availability for Marketing Campaigns
Scalability and Availability for Marketing CampaignsAmazon Web Services
 
Architecting with AWS Sydney Customer Appreciation Day
Architecting with AWS Sydney Customer Appreciation DayArchitecting with AWS Sydney Customer Appreciation Day
Architecting with AWS Sydney Customer Appreciation DayAmazon Web Services
 
Security and Privacy in the AWS Cloud - AWS India Summit 2012
Security and Privacy in the AWS Cloud - AWS India Summit 2012Security and Privacy in the AWS Cloud - AWS India Summit 2012
Security and Privacy in the AWS Cloud - AWS India Summit 2012Amazon Web Services
 
RightScale Sydney Customer Appreciation Day
RightScale Sydney Customer Appreciation DayRightScale Sydney Customer Appreciation Day
RightScale Sydney Customer Appreciation DayAmazon Web Services
 

Destacado (20)

Three Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active ArchiveThree Steps to Modern Media Asset Management with Active Archive
Three Steps to Modern Media Asset Management with Active Archive
 
Digital Media Ingest and Storage Options on AWS
Digital Media Ingest and Storage Options on AWSDigital Media Ingest and Storage Options on AWS
Digital Media Ingest and Storage Options on AWS
 
SUN TV NETWORK LIMITED
SUN TV NETWORK LIMITEDSUN TV NETWORK LIMITED
SUN TV NETWORK LIMITED
 
ximena araneda - The Next Generation MAM Systems
ximena araneda - The Next Generation MAM Systemsximena araneda - The Next Generation MAM Systems
ximena araneda - The Next Generation MAM Systems
 
AWS Customer Presentation - Smugmug
AWS Customer Presentation - SmugmugAWS Customer Presentation - Smugmug
AWS Customer Presentation - Smugmug
 
ARC304 Solutions in Action - AWS re: Invent 2012
ARC304 Solutions in Action - AWS re: Invent 2012ARC304 Solutions in Action - AWS re: Invent 2012
ARC304 Solutions in Action - AWS re: Invent 2012
 
AWS Customer Presentation - Turbo10
AWS Customer Presentation - Turbo10AWS Customer Presentation - Turbo10
AWS Customer Presentation - Turbo10
 
Choosing the Right Data Storage - Carlos Conde - AWS Summit Paris
Choosing the Right Data Storage - Carlos Conde - AWS Summit ParisChoosing the Right Data Storage - Carlos Conde - AWS Summit Paris
Choosing the Right Data Storage - Carlos Conde - AWS Summit Paris
 
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012
TLS304 Getting Productive with the AWS SDK for Ruby - AWS re: Invent 2012
 
AWS Update | London - Amazon Glacier
AWS Update | London - Amazon GlacierAWS Update | London - Amazon Glacier
AWS Update | London - Amazon Glacier
 
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012
MBL101 Distributing through Appstore and Kindle Fire - AWS re: Invent 2012
 
AWS Customer Presentation - MessageSling
AWS Customer Presentation - MessageSlingAWS Customer Presentation - MessageSling
AWS Customer Presentation - MessageSling
 
AWS Customer Presentation - Nutsie
AWS Customer Presentation -  Nutsie AWS Customer Presentation -  Nutsie
AWS Customer Presentation - Nutsie
 
AWS Customer Presentation - Mahalo
AWS Customer Presentation - Mahalo AWS Customer Presentation - Mahalo
AWS Customer Presentation - Mahalo
 
AWS Customer Presentation - HotPads
AWS Customer Presentation - HotPadsAWS Customer Presentation - HotPads
AWS Customer Presentation - HotPads
 
Scalability and Availability for Marketing Campaigns
Scalability and Availability for Marketing CampaignsScalability and Availability for Marketing Campaigns
Scalability and Availability for Marketing Campaigns
 
Dr. Werner Vogels Keynote
Dr. Werner Vogels KeynoteDr. Werner Vogels Keynote
Dr. Werner Vogels Keynote
 
Architecting with AWS Sydney Customer Appreciation Day
Architecting with AWS Sydney Customer Appreciation DayArchitecting with AWS Sydney Customer Appreciation Day
Architecting with AWS Sydney Customer Appreciation Day
 
Security and Privacy in the AWS Cloud - AWS India Summit 2012
Security and Privacy in the AWS Cloud - AWS India Summit 2012Security and Privacy in the AWS Cloud - AWS India Summit 2012
Security and Privacy in the AWS Cloud - AWS India Summit 2012
 
RightScale Sydney Customer Appreciation Day
RightScale Sydney Customer Appreciation DayRightScale Sydney Customer Appreciation Day
RightScale Sydney Customer Appreciation Day
 

Similar a MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012

AWS Well Architected-Info Session WeCloudData
AWS Well Architected-Info Session WeCloudDataAWS Well Architected-Info Session WeCloudData
AWS Well Architected-Info Session WeCloudDataWeCloudData
 
Backup and archiving in the aws cloud
Backup and archiving in the aws cloudBackup and archiving in the aws cloud
Backup and archiving in the aws cloudAmazon Web Services
 
Scaling the Platform for Your Startup - Startup Talks June 2015
Scaling the Platform for Your Startup - Startup Talks June 2015Scaling the Platform for Your Startup - Startup Talks June 2015
Scaling the Platform for Your Startup - Startup Talks June 2015Amazon Web Services
 
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...Amazon Web Services
 
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...Amazon Web Services
 
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...Amazon Web Services
 
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon Glacier
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon GlacierSRV403 Deep Dive on Object Storage: Amazon S3 and Amazon Glacier
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon GlacierAmazon Web Services
 
ENT306 Migrating large Scale Data Sets to the Cloud
ENT306 Migrating large Scale Data Sets to the CloudENT306 Migrating large Scale Data Sets to the Cloud
ENT306 Migrating large Scale Data Sets to the CloudAmazon Web Services
 
Demystifying Storage on AWS | AWS Public Sector Summit 2017
Demystifying Storage on AWS | AWS Public Sector Summit 2017Demystifying Storage on AWS | AWS Public Sector Summit 2017
Demystifying Storage on AWS | AWS Public Sector Summit 2017Amazon Web Services
 
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)Amazon Web Services
 
Choosing the right data storage in the Cloud.
Choosing the right data storage in the Cloud. Choosing the right data storage in the Cloud.
Choosing the right data storage in the Cloud. Amazon Web Services
 
Moving Your Media Supply Chain to the AWS Cloud
Moving Your Media Supply Chain to the AWS CloudMoving Your Media Supply Chain to the AWS Cloud
Moving Your Media Supply Chain to the AWS CloudAmazon Web Services
 
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...Optimizing Data Management Using AWS Storage and Data Migration Products | AW...
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...Amazon Web Services
 
Strategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageStrategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageAmazon Web Services
 
Scaling up to your first 10 million users - Pop-up Loft Tel Aviv
Scaling up to your first 10 million users - Pop-up Loft Tel AvivScaling up to your first 10 million users - Pop-up Loft Tel Aviv
Scaling up to your first 10 million users - Pop-up Loft Tel AvivAmazon Web Services
 
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big DataAmazon Web Services
 
Data Migration Using AWS Snowball, Snowball Edge & Snowmobile
Data Migration Using AWS Snowball, Snowball Edge & SnowmobileData Migration Using AWS Snowball, Snowball Edge & Snowmobile
Data Migration Using AWS Snowball, Snowball Edge & SnowmobileAmazon Web Services
 

Similar a MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012 (20)

Create cloud service on AWS
Create cloud service on AWSCreate cloud service on AWS
Create cloud service on AWS
 
AWS Well Architected-Info Session WeCloudData
AWS Well Architected-Info Session WeCloudDataAWS Well Architected-Info Session WeCloudData
AWS Well Architected-Info Session WeCloudData
 
Backup and archiving in the aws cloud
Backup and archiving in the aws cloudBackup and archiving in the aws cloud
Backup and archiving in the aws cloud
 
Scaling the Platform for Your Startup - Startup Talks June 2015
Scaling the Platform for Your Startup - Startup Talks June 2015Scaling the Platform for Your Startup - Startup Talks June 2015
Scaling the Platform for Your Startup - Startup Talks June 2015
 
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...
[AWS LA Media & Entertainment Event 2015]: Digital Media Ingest & Storage Opt...
 
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
 
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...
AWS May Webinar Series - Getting Started: Storage with Amazon S3 and Amazon G...
 
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon Glacier
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon GlacierSRV403 Deep Dive on Object Storage: Amazon S3 and Amazon Glacier
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon Glacier
 
ENT306 Migrating large Scale Data Sets to the Cloud
ENT306 Migrating large Scale Data Sets to the CloudENT306 Migrating large Scale Data Sets to the Cloud
ENT306 Migrating large Scale Data Sets to the Cloud
 
Demystifying Storage on AWS | AWS Public Sector Summit 2017
Demystifying Storage on AWS | AWS Public Sector Summit 2017Demystifying Storage on AWS | AWS Public Sector Summit 2017
Demystifying Storage on AWS | AWS Public Sector Summit 2017
 
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)
AWS re:Invent 2016: High Performance Cinematic Production in the Cloud (MAE304)
 
Ingest and storage options
Ingest and storage optionsIngest and storage options
Ingest and storage options
 
Choosing the right data storage in the Cloud.
Choosing the right data storage in the Cloud. Choosing the right data storage in the Cloud.
Choosing the right data storage in the Cloud.
 
Moving Your Media Supply Chain to the AWS Cloud
Moving Your Media Supply Chain to the AWS CloudMoving Your Media Supply Chain to the AWS Cloud
Moving Your Media Supply Chain to the AWS Cloud
 
Aws storage for media overview
Aws storage for media overview Aws storage for media overview
Aws storage for media overview
 
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...Optimizing Data Management Using AWS Storage and Data Migration Products | AW...
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...
 
Strategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageStrategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud Storage
 
Scaling up to your first 10 million users - Pop-up Loft Tel Aviv
Scaling up to your first 10 million users - Pop-up Loft Tel AvivScaling up to your first 10 million users - Pop-up Loft Tel Aviv
Scaling up to your first 10 million users - Pop-up Loft Tel Aviv
 
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
 
Data Migration Using AWS Snowball, Snowball Edge & Snowmobile
Data Migration Using AWS Snowball, Snowball Edge & SnowmobileData Migration Using AWS Snowball, Snowball Edge & Snowmobile
Data Migration Using AWS Snowball, Snowball Edge & Snowmobile
 

Más de Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Más de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

MED201 Media Ingest and Storage Solutions with AWS - AWS re: Invent 2012

  • 5. UNIQUE CUSTOMER OBJECTS
  • 6. PEAK TRANSACTIONS PER SECOND
  • 7. 99.99% vs. 99.999999999% durability of objects over a given year
  • 8. Customer Decides Where Applications and Data Reside
  • 10. • Disaster Recovery • Backup Region 2 Region 1 • Distribution • Archive • Load for Elastic MapReduce • Migrate / Sync www.attunitycloudbeam.com Enterprise • SaaS-based solution • Simplicity, “click and replicate” • Speed, optimized and elastic transfer • Safe, secure and guaranteed delivery
  • 12. PER GB / MONTH
  • 13. PER TB / YEAR
  • 17. Archive after 30 days logs/file1 logs/file2 logs/file3 logs Amazon Glacier My S3 bucket
  • 18. Expire after 365 days logs/file1 logs/file2 logs logs/file3 Amazon Glacier My S3 bucket
  • 19. Storage Options Amazon Reduced Data Glacier Redundancy Storage Standard Storage Amazon Glacier Amazon S3
  • 23. HSM with AWS Amazon Amazon SAN S3 AWS Cloud Glacier Corporate Data Center versus Traditional Approach to HSM offsite tape SAN tier 2 disk storage storage backup Corporate Data Center
  • 24. Compliance with AWS O/S image Amazon Amazon S3 AWS Cloud Glacier versus Traditional Approach off-site tape O/S image disk storage backup Corporate Data Center
  • 31. 5 HTTP – 2 fasp 3 multipart 4 Parallel Transcoding fasp 5 14 instances: 3 min Herndon, VA 1 6 1. Video broadcast capture 2. High-speed upload Direct-to-S3 3. Scale out parallel transcode ` 4. Deliver back to S3 5. High-speed download from S3 to UFC 6. Insert into CMS for streaming to mobile devices
  • 35. What is Imgur? • A simple image sharer • Has the most viral images on the Internet • Anyone can upload as many images as they want – without an account • 2,000,000 images uploaded per day • That’s 23 images per second • Can be embedded and shared on any site
  • 36. The greatest image site. Full of the all the wonders and magic of the interwebs. Be forewarned, time has been known to quicken in this realm. “I spent half a day on Imgur, and it was the greatest 6 hours of my life.” - Urban Dictionary
  • 38. • Started as a side project while at Ohio University • Redditors needed a place to host their images • Organically grew into a business • Alan was the only developer for 3 years • Moved to San Francisco • Now a team of 7 • (600 million pageviews per engineer)
  • 39. Every month. There are: • 11 minutes average visit duration • 2.9 billion page views • 11 pages per visit • 38 billion image views (images loaded) • 46th biggest site in the US • 54 million unique visitors – (according to alexa.com) • 4.7 petabytes of bandwidth used • 600 million objects stored in S3 • 62 million images uploaded * All data as of Nov 2012
  • 40. • Pageviews are growing 15% every month. • How are we able to support this kind of growth?
  • 41. User make a request for an image (Don’t do this!)
  • 42. User make a request for an image
  • 43. User make a request for an image
  • 46. • Site traffic is increasing more than ever. How many more servers do we need? • Hardware failures • Tweaking every little thing is really hard and easy to get wrong, but necessary • There’s only one man doing all this; how can we make his life easier, while scaling the site at the same time?
  • 47. Autoscaling is awesome • Automated DB backups are awesome • Security features are awesome • Much easier to manage in the long run • Because everything’s managed, you require less admins to look over everything all the time • AWS has managed solutions for all the core services your website needs (server, database, cache, backups, security, etc.)
  • 48. • Lots of new stuff to learn and set up • Possible downtime during migration • Very time consuming at first because you’re reconfiguring your entire stack
  • 49. AWS has a lot of services; find out which ones can work for you and how • Use the price calculator: http://calculator.s3.amazonaws.com/calc5.html • Read the docs: http://aws.amazon.com • Set up a test environment • Install the AWS SDK • Call AWS if you have questions. • (You don’t need AWS Support to call in) • Start coding!
  • 50. How do you get all your data to S3? • Duplicate writes: 1 to native, 1 to S3. • Upload all your data to S3 in parallel.We had 12 background processes running around the clock all uploading a different subset of data to S3 – it took 2 weeks to finish. • No need to store more than one copy • Turn on versioning for even more protection • Very similar process for Amazon RDS
  • 51. • There’s no web interface • Have to do everything from command line • Confusing terminology • Hard to verify that it’s working as intended • But in the end, it’s amazing • If you’re not using it, you’d better have a really good reason why
  • 52. EC2: • Maximum performance with RAID0 Elastic Block Store • RAID0 EBS requires a pretty significant amount of maintenance overhead • Have to come up with your own backup plan • RDS: • Will provide very good performance out of the box (but not maximum) • Management console is fantastic • Easy to upgrade instances • High availability and read-only slaves are a click away • Managed service, which makes it more expensive • If you enjoy tuning every last little bit for maximum performance, then you can consider EC2 + EBS RAID 0 • Still on the fence? Go with RDS
  • 53. • There’s no access to the underlying file system • Migrating requires a dump and an import of your data, which is extremely time consuming for large databases • No access to the logs when things break • We were able to do it live – without taking the site down – but with lots of headaches
  • 56. Wed (1:00 p.m.–1:50 p.m.) MED203: Scalable Media Processing with AWS • Wed (2:05 p.m.–2:55 p.m.) MED202: Netflix’s Transcoding Transformation • Wed (3:25 p.m.–4:15 p.m.) MED303: Addressing Security in Media Workflow • Thu (10:30 a.m.–11:20 a.m.) STG205: Amazon S3: Reduce costs, save time, and better protect your data • Thu (11:35 a.m.–12:25 p.m.) STG203: Cloud Storage War Stories: From the front lines of some of the biggest battles • Thu (4:05 p.m.–4:55 p.m.) STG302: Archive in the Cloud with Amazon Glacier http://aws.amazon.com/s3/ http://aws.amazon.com/glacier/faqs/ http://aws.amazon.com/digital-media/ • Wed (1:00 p.m.–1:50 p.m.) STG201: Understanding AWS Storage Options
  • 57. We are sincerely eager to tweet #reinvent hear your feedback on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance.