SlideShare una empresa de Scribd logo
1 de 55
DAT101 Understanding AWS Database Options - AWS re: Invent 2012
AWS Database Options and Decision Factors

Best Practice Tips and Techniques

   • Optimizing for Manageability and Scale  Edmodo

   • Optimizing for App Velocity and Scale  Obama for America

   • Leveraging YesSQL and NoSQL  BrandVerity

Q&A
Before We Begin
Easily and rapidly analyze
petabytes of data

1/10 the cost of traditional
data warehouses

Automated deployment &
administration

Compatible with popular
BI tools
Common BI Tools

                                                            Choose from 16TB local disk / 128 GB
                            JDBC/ODBC                       RAM or 2TB local disk / 16GB RAM
                                                            nodes

                                    Leader
                                    Node
                                                            Configure up to 100 nodes for up to
                                                            1.6 Pb
                  10GigE Mesh


Amazon Redshift                                             Data stored in columnar format for 10X
                  Compute          Compute        Compute
                                                            I/O efficiencies and fast queries
                  Node             Node           Node




                                                            Query with standard SQL and
                                                            JDBC/ODBC
Your
Amazon Redshift                        BI Tools

                    ODBC /
                    JDBC
                  PostgreSQL drivers
1. Zero to App in ____ Minutes

2. Zero to Millions of users in ____ Days

3. Zero to “IPO” in ____ Months
1. Zero to App in ____ Minutes

2. Zero to Millions of users in ____ Days

3. Zero to “IPO” in ____ Months
Focus on your App
Load balancer

Application tier


Database tier
Load balancer
                                           Security, Scale, Availability…

                                                                     Application tier

                         Security, Innovation, Scale, Performance, Availability…

                                                                      Database tier

Security, Innovation, Scale, Transactions, Performance, Durability, Availability, Skills..
SQL              NoSQL



Do-it Yourself   Fully
                 Managed
                             Not available



                             on AWS

Low Cost         High Cost
SQL              NoSQL



Do-it Yourself   Fully
                 Managed
SQL               NoSQL



Do-it Yourself    Fully
                  Managed
MySQL            MySQL
Oracle           Oracle
SQL Server       SQL Server
MariaDB
Postgres
…
SQL               NoSQL



  Do-it Yourself    Fully
                    Managed

MongoDB            DynamoDB
Cassandra          ElastiCache
Redis              SimpleDB
Memcache
Should I use
Should I use SQL         MySQL on EC2 or
  or NoSQL?                  RDS?




 Should I use
  MongoDB,
                   ?   Should I use Redis,
 Cassandra, or           Memcache, or
 DynamoDB?               ElastiCache?
What are my
What are my scale           transactional and
and latency needs?         consistency needs?




   What are my
                      ?   What are my time to
read/write, storage       market and server
 and IOPS needs?           control needs?
Factors                SQL                                  NoSQL

Application            • App with complex business logic?   • Web app with lots of users?

Transactions           • Complex txns, joins, updates?      • Simple data model, updates, queries?

Scale                  • Developer managed                  • Automatic, on-demand scaling

Performance            • Developer architected              • Consistent, high performance at scale

Availability           • Architected for fail-over          • Seamless and transparent

Core Skills            • SQL + Java/Ruby/Python/PhP         • NoSQL + Java/Ruby/Python/PhP



               Best of both worlds: Possible to Use SQL and NoSQL models in one App
Factors             Do it Yourself (DIY)                     Fully Managed

Replication         • Granular, app managed                  • Transparent and configured

Monitoring          • Specific agents and custom             • Automated and API driven

Security            • Root access, custom configs            • Hardened by the service

Resources           • Requires more DBA resources and time   • Requires less DBA resources and time

Time to market      • Sophistication vs. speed               • Rapid iteration

Core Skills         • Systems, databases, monitoring         • Applications, User focused


              Best of both worlds: Possible to manage different tiers differently
Amazon RDS is a fully managed SQL database service.

    Choice of Database engines
    Simple to deploy and scale
    Reliable and cost effective
    Without any operational burden.
Migration
                                           Backup and recovery
Schema design                              Patching
Query construction                         Configuration
Query optimization                         Software upgrades
                                           Storage upgrades
                                           Frequent server upgrades
    Focus on the “innovation”
                                           Hardware crash
                    Off load the “administration”
 Multiple databases per instance
 Standard user accounts
 Connect and query using common MySQL tools & drivers
 Tune engine parameters
 Import and export data using standard MySQL tools (mysqldump)
 Diagnostics
 Native MySQL replication
 SSL for encryption over the wire
 Monitor metrics

 Shell, super user or direct file system access (Think security!)
ElastiCache is a fully managed Memcache
caching service.

Easy to set up and operate
Scale cache clusters with push button ease
Ultra fast response time for read scaling
Without any operational burden.
Amazon DynamoDB is a fully managed NoSQL
database service.

Store and retrieve any amount of data
Scale throughput to millions of IO
Single digit millisecond latencies
Without any operational burden.
CreateTable    PutItem

                           UpdateTable
                                              GetItem

                            DeleteTable
                                             UpdateItem
                                                             “Select”, “insert”, “update”
                           DescribeTable                     items
Manage tables
                                             DeleteItem
                             ListTables

                                            BatchGetItem
                               Query
                                                              Bulk select or update
 Query specific items OR        Scan        BatchWriteItem    (max 1MB)
 scan the full table
So, what are the tips and techniques for
successful deployments?
Educates millions of students    Amazon
                                  EC2
                                 Amazon
                                DynamoDB
                                 Amazon
Reaches millions of citizens    Elasticache

                                 Amazon
                                  RDS

                                 Amazon
Analyzes billions of Ads           S3
Kimo
Educates millions of students
                                Rosenbaum


Reaches millions of citizens


Analyzes billions of Ads
Kimo Rosenbaum – Data Architect, Edmodo
Where learning happens.



                            Kimo Rosenbaum
                          kimo@edmodo.com
                          AWS re: Invent 2012
Learning 101

•   Largest, fastest growing social platform for education
•   Secure learning network for teachers and students
•   Browser, iOS, Android
•   Free for teachers and students
Stats 101

•   100,000 schools
•   14 million users
•   7 million new users in the last year
•   1 million visits daily
Web


                                                                      Instance
                                                               Auto scaling Group
                                                                                                                          Amazon CloudWatch
Amazon Route 53           Elastic Load
                           Balancer
                                                       Cache                           Cache


                                                        Instance                       Instance




Amazon Cloudfront                                                                                                                Instances
                          Amazon S3

                                           RDS DB Instance         RDS DB Instance           RDS DB Instance
                                            Read Replica            Read Replica              Read Replica


                                                                   Availability Zone




     RDS DB Instance                               RDS DB Instance                                             RDS DB Instance
      Read Replica     MySQL DB Instance            Read Replica            MySQL DB Instance                   Read Replica       MySQL DB Instance
DBA 101

•   Restore from snapshot
•   Replica creation
•   Parameter tuning
•   Metrics collection
•   Know your app/data
Educates millions of students


                                  Jay
Reaches millions of citizens
                                Edwards


Analyzes billions of Ads
Jay Edwards – Database Engineer, Obama Campaign
Me.



• Twitter: First dedicated DBA
• OFA: Lead Database Engineer
• PalominoDB: CTO & VP/Operations
Obama for America.


• Technically sophisticated for a campaign
      • Not “web-scale”
• Hockey-stick++ growth
• Downtime hurts. A lot…really, really, really a lot.
Hockey-stick++
OFA Architecture    RDS Read Replica




      ElastiCache
                              RDS with

  DynamoDB                    Multi-AZ




                                ELB
Problems!


• You always need more databases
     • OFA had 24+ schemas & 100+ RDS instances
• You never have enough DBAs
     • OFA had 1 – 2 x 0.5 fulltime MySQL DBAs
Why RDS?


• Makes operational issues very easy
     • Need more replicas? BAM!
     • Upsize hardware? KAPOW!
     • Point in time restore? BIF!
Why not RDS?

• Hardware cap (vertical v. horizontal)
• Sophisticated use-cases
     • Frequent topology changes
     • Multi-region replication (on their roadmap)
• DBAs need busy work
Educates millions of students


Reaches millions of citizens


                                Andy
Analyzes billions of Ads
                                Skalet
Andy Skalet - CTO, BrandVerity
Managed Services Bias
New Products/Markets – YesSQL!
Big Data? Cast your problem
AWS Options
Case Study: Crawl history
DAT101 Understanding AWS Database Options - AWS re: Invent 2012
andy@brandverity.com
• Managed services let you focus on creating value

• Amazon S3 - Very robust, handles large items, but you filter

• Amazon DynamoDB - Extremely fast, scalable, good value
  • Must cast your problem as kvs or key + range

• Amazon RDS - MySQL, without the headaches

• Amazon ElastiCache - As memcached, fast kvs for small data

• Multi column queries on big data?
  • Looking forward to the AWS solution
Thank you
Free   aws.amazon.com/dynamodb
Tier
       aws.amazon.com/rds
       aws.amazon.com/elasticache
       raghavas@amazon
We are sincerely eager to
 hear your feedback on this
presentation and on re:Invent.

 Please fill out an evaluation
   form when you have a
            chance.

Más contenido relacionado

La actualidad más candente

AWS Update | London - Overview of New Releases
AWS Update | London - Overview of New ReleasesAWS Update | London - Overview of New Releases
AWS Update | London - Overview of New ReleasesAmazon Web Services
 
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...
2011 State of the Cloud:  A Year's Worth of Innovation in 30 Minutes - Jinesh...2011 State of the Cloud:  A Year's Worth of Innovation in 30 Minutes - Jinesh...
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...Amazon Web Services
 
My sql 5.6_replwebinar_may12
My sql 5.6_replwebinar_may12My sql 5.6_replwebinar_may12
My sql 5.6_replwebinar_may12Mat Keep
 
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...Amazon Web Services
 
Cost Optimisation with Amazon Web Services
 Cost Optimisation with Amazon Web Services Cost Optimisation with Amazon Web Services
Cost Optimisation with Amazon Web ServicesAmazon Web Services
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connectAdrian Cockcroft
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformAdrian Cockcroft
 
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013Amazon Web Services
 
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...IndicThreads
 
A Step By Step Guide To Put DB2 On Amazon Cloud
A Step By Step Guide To Put DB2 On Amazon CloudA Step By Step Guide To Put DB2 On Amazon Cloud
A Step By Step Guide To Put DB2 On Amazon CloudDeepak Rao
 
Next Generation of Storage Sydney Customer Appreciation Day
Next Generation of Storage Sydney Customer Appreciation DayNext Generation of Storage Sydney Customer Appreciation Day
Next Generation of Storage Sydney Customer Appreciation DayAmazon Web Services
 
Netflix Cloud Platform Building Blocks
Netflix Cloud Platform Building BlocksNetflix Cloud Platform Building Blocks
Netflix Cloud Platform Building BlocksSudhir Tonse
 
AWS Webcast - Introducing Amazon RDS for PostgreSQL
AWS Webcast - Introducing Amazon RDS for PostgreSQLAWS Webcast - Introducing Amazon RDS for PostgreSQL
AWS Webcast - Introducing Amazon RDS for PostgreSQLAmazon Web Services
 
Cloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsCloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsMark Slingsby
 
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデートAmazon Web Services Japan
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Adrian Cockcroft
 
Cloud computing with AWS
Cloud computing with AWS Cloud computing with AWS
Cloud computing with AWS ikanow
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAmazon Web Services
 

La actualidad más candente (20)

AWS Update | London - Overview of New Releases
AWS Update | London - Overview of New ReleasesAWS Update | London - Overview of New Releases
AWS Update | London - Overview of New Releases
 
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...
2011 State of the Cloud:  A Year's Worth of Innovation in 30 Minutes - Jinesh...2011 State of the Cloud:  A Year's Worth of Innovation in 30 Minutes - Jinesh...
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...
 
My sql 5.6_replwebinar_may12
My sql 5.6_replwebinar_may12My sql 5.6_replwebinar_may12
My sql 5.6_replwebinar_may12
 
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...Keynote: Your Future With Cloud Computing - Dr. Werner Vogels  - AWS Summit 2...
Keynote: Your Future With Cloud Computing - Dr. Werner Vogels - AWS Summit 2...
 
Cost Optimisation with Amazon Web Services
 Cost Optimisation with Amazon Web Services Cost Optimisation with Amazon Web Services
Cost Optimisation with Amazon Web Services
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source Platform
 
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013
Introducing Amazon RDS for PostgreSQL (DAT210) | AWS re:Invent 2013
 
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
 
A Step By Step Guide To Put DB2 On Amazon Cloud
A Step By Step Guide To Put DB2 On Amazon CloudA Step By Step Guide To Put DB2 On Amazon Cloud
A Step By Step Guide To Put DB2 On Amazon Cloud
 
Next Generation of Storage Sydney Customer Appreciation Day
Next Generation of Storage Sydney Customer Appreciation DayNext Generation of Storage Sydney Customer Appreciation Day
Next Generation of Storage Sydney Customer Appreciation Day
 
Netflix Cloud Platform Building Blocks
Netflix Cloud Platform Building BlocksNetflix Cloud Platform Building Blocks
Netflix Cloud Platform Building Blocks
 
AWS Webcast - Introducing Amazon RDS for PostgreSQL
AWS Webcast - Introducing Amazon RDS for PostgreSQLAWS Webcast - Introducing Amazon RDS for PostgreSQL
AWS Webcast - Introducing Amazon RDS for PostgreSQL
 
Cloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsCloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web Apps
 
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート
20191218 AWS Black Belt Online Seminar AWSのマネジメント&ガバナンス サービスアップデート
 
Keynote from Werner Vogels
Keynote from Werner VogelsKeynote from Werner Vogels
Keynote from Werner Vogels
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
 
Cloud computing with AWS
Cloud computing with AWS Cloud computing with AWS
Cloud computing with AWS
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloud
 
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
 

Destacado

An introduction to cloud computing with Amazon Web Services and MongoDB
An introduction to cloud computing with Amazon Web Services and MongoDBAn introduction to cloud computing with Amazon Web Services and MongoDB
An introduction to cloud computing with Amazon Web Services and MongoDBSamuel Demharter
 
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014Amazon Web Services
 
Hadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the HerdHadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the HerdIBM Analytics
 
AWS Webcast - Redshift Overview and New Features
AWS Webcast - Redshift Overview and New Features AWS Webcast - Redshift Overview and New Features
AWS Webcast - Redshift Overview and New Features Amazon Web Services
 
Compare DynamoDB vs. MongoDB
Compare DynamoDB vs. MongoDBCompare DynamoDB vs. MongoDB
Compare DynamoDB vs. MongoDBAmar Das
 
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013Amazon Web Services
 
Cloudera Showcase: SQL-on-Hadoop
Cloudera Showcase: SQL-on-HadoopCloudera Showcase: SQL-on-Hadoop
Cloudera Showcase: SQL-on-HadoopCloudera, Inc.
 
Bi on Big Data - Strata 2016 in London
Bi on Big Data - Strata 2016 in LondonBi on Big Data - Strata 2016 in London
Bi on Big Data - Strata 2016 in LondonDremio Corporation
 
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013Amazon Web Services
 
Daniel Abadi HadoopWorld 2010
Daniel Abadi HadoopWorld 2010Daniel Abadi HadoopWorld 2010
Daniel Abadi HadoopWorld 2010Daniel Abadi
 
BCBS 239 - Risk Data Adequacy
BCBS 239 - Risk Data AdequacyBCBS 239 - Risk Data Adequacy
BCBS 239 - Risk Data Adequacynikatmalik
 
AWS re:Invent 2016 recap (part 2)
AWS re:Invent 2016 recap (part 2) AWS re:Invent 2016 recap (part 2)
AWS re:Invent 2016 recap (part 2) Julien SIMON
 
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)Amazon Web Services
 

Destacado (17)

An introduction to cloud computing with Amazon Web Services and MongoDB
An introduction to cloud computing with Amazon Web Services and MongoDBAn introduction to cloud computing with Amazon Web Services and MongoDB
An introduction to cloud computing with Amazon Web Services and MongoDB
 
El uso de la c
El uso de la cEl uso de la c
El uso de la c
 
Intro to AWS: Database Services
Intro to AWS: Database ServicesIntro to AWS: Database Services
Intro to AWS: Database Services
 
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014
 
Hadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the HerdHadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the Herd
 
AWS Webcast - Redshift Overview and New Features
AWS Webcast - Redshift Overview and New Features AWS Webcast - Redshift Overview and New Features
AWS Webcast - Redshift Overview and New Features
 
Compare DynamoDB vs. MongoDB
Compare DynamoDB vs. MongoDBCompare DynamoDB vs. MongoDB
Compare DynamoDB vs. MongoDB
 
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013
Introduction to Amazon Redshift and What's Next (DAT103) | AWS re:Invent 2013
 
Cloudera Showcase: SQL-on-Hadoop
Cloudera Showcase: SQL-on-HadoopCloudera Showcase: SQL-on-Hadoop
Cloudera Showcase: SQL-on-Hadoop
 
Bi on Big Data - Strata 2016 in London
Bi on Big Data - Strata 2016 in LondonBi on Big Data - Strata 2016 in London
Bi on Big Data - Strata 2016 in London
 
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
 
Daniel Abadi HadoopWorld 2010
Daniel Abadi HadoopWorld 2010Daniel Abadi HadoopWorld 2010
Daniel Abadi HadoopWorld 2010
 
BCBS 239 - Risk Data Adequacy
BCBS 239 - Risk Data AdequacyBCBS 239 - Risk Data Adequacy
BCBS 239 - Risk Data Adequacy
 
AWS re:Invent 2016 recap (part 2)
AWS re:Invent 2016 recap (part 2) AWS re:Invent 2016 recap (part 2)
AWS re:Invent 2016 recap (part 2)
 
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)
 
What is Big Data?
What is Big Data?What is Big Data?
What is Big Data?
 
Big data ppt
Big  data pptBig  data ppt
Big data ppt
 

Similar a DAT101 Understanding AWS Database Options - AWS re: Invent 2012

Building Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYCBuilding Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYCAmazon Web Services
 
AWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases OptionsAWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases OptionsAmazon Web Services
 
Dynamo DB & RDS Deep Dive - AWS India Summit 2012
Dynamo DB & RDS Deep Dive - AWS India Summit 2012Dynamo DB & RDS Deep Dive - AWS India Summit 2012
Dynamo DB & RDS Deep Dive - AWS India Summit 2012Amazon Web Services
 
Relational Databases Redefined with AWS
Relational Databases Redefined with AWSRelational Databases Redefined with AWS
Relational Databases Redefined with AWSAmazon Web Services
 
AWS Cloud Kata 2014 | Jakarta - Startup Best Practices
AWS Cloud Kata 2014 | Jakarta - Startup Best PracticesAWS Cloud Kata 2014 | Jakarta - Startup Best Practices
AWS Cloud Kata 2014 | Jakarta - Startup Best PracticesAmazon Web Services
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerMichael Rys
 
Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Amazon Web Services
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Amazon Web Services
 
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...Amazon Web Services
 
Introducing Database Offerings on AWS - Technical 101
Introducing Database Offerings on AWS - Technical 101Introducing Database Offerings on AWS - Technical 101
Introducing Database Offerings on AWS - Technical 101Amazon Web Services
 
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWSTransformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWSAmazon Web Services LATAM
 
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...Amazon Web Services
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinAmazon Web Services
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinIan Massingham
 
ENT309 scaling up to your first 10 million users
ENT309 scaling up to your first 10 million usersENT309 scaling up to your first 10 million users
ENT309 scaling up to your first 10 million usersAmazon Web Services
 
Cloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebCloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebjineshvaria
 
Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Amazon Web Services
 
Minnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraMinnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraJeff Bollinger
 

Similar a DAT101 Understanding AWS Database Options - AWS re: Invent 2012 (20)

Understanding Database Options
Understanding Database OptionsUnderstanding Database Options
Understanding Database Options
 
Building Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYCBuilding Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYC
 
AWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases OptionsAWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases Options
 
Dynamo DB & RDS Deep Dive - AWS India Summit 2012
Dynamo DB & RDS Deep Dive - AWS India Summit 2012Dynamo DB & RDS Deep Dive - AWS India Summit 2012
Dynamo DB & RDS Deep Dive - AWS India Summit 2012
 
High Performance Databases
High Performance DatabasesHigh Performance Databases
High Performance Databases
 
Relational Databases Redefined with AWS
Relational Databases Redefined with AWSRelational Databases Redefined with AWS
Relational Databases Redefined with AWS
 
AWS Cloud Kata 2014 | Jakarta - Startup Best Practices
AWS Cloud Kata 2014 | Jakarta - Startup Best PracticesAWS Cloud Kata 2014 | Jakarta - Startup Best Practices
AWS Cloud Kata 2014 | Jakarta - Startup Best Practices
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL Server
 
Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2
 
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...
 
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...
A Petascale Database for Large-Scale Neuroscience Powered by Serverless Advan...
 
Introducing Database Offerings on AWS - Technical 101
Introducing Database Offerings on AWS - Technical 101Introducing Database Offerings on AWS - Technical 101
Introducing Database Offerings on AWS - Technical 101
 
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWSTransformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS
Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS
 
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...
The Boss: A Petascale Database for Large-Scale Neuroscience, Powered by Serve...
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
 
ENT309 scaling up to your first 10 million users
ENT309 scaling up to your first 10 million usersENT309 scaling up to your first 10 million users
ENT309 scaling up to your first 10 million users
 
Cloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWebCloud Architectures - Jinesh Varia - GrepTheWeb
Cloud Architectures - Jinesh Varia - GrepTheWeb
 
Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)Why Scale Matters and How the Cloud is Really Different (at scale)
Why Scale Matters and How the Cloud is Really Different (at scale)
 
Minnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraMinnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with Cassandra
 

Más de Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Más de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

DAT101 Understanding AWS Database Options - AWS re: Invent 2012

  • 2. AWS Database Options and Decision Factors Best Practice Tips and Techniques • Optimizing for Manageability and Scale  Edmodo • Optimizing for App Velocity and Scale  Obama for America • Leveraging YesSQL and NoSQL  BrandVerity Q&A
  • 4. Easily and rapidly analyze petabytes of data 1/10 the cost of traditional data warehouses Automated deployment & administration Compatible with popular BI tools
  • 5. Common BI Tools Choose from 16TB local disk / 128 GB JDBC/ODBC RAM or 2TB local disk / 16GB RAM nodes Leader Node Configure up to 100 nodes for up to 1.6 Pb 10GigE Mesh Amazon Redshift Data stored in columnar format for 10X Compute Compute Compute I/O efficiencies and fast queries Node Node Node Query with standard SQL and JDBC/ODBC
  • 6. Your Amazon Redshift BI Tools ODBC / JDBC PostgreSQL drivers
  • 7. 1. Zero to App in ____ Minutes 2. Zero to Millions of users in ____ Days 3. Zero to “IPO” in ____ Months
  • 8. 1. Zero to App in ____ Minutes 2. Zero to Millions of users in ____ Days 3. Zero to “IPO” in ____ Months
  • 11. Load balancer Security, Scale, Availability… Application tier Security, Innovation, Scale, Performance, Availability… Database tier Security, Innovation, Scale, Transactions, Performance, Durability, Availability, Skills..
  • 12. SQL NoSQL Do-it Yourself Fully Managed Not available  on AWS Low Cost High Cost
  • 13. SQL NoSQL Do-it Yourself Fully Managed
  • 14. SQL NoSQL Do-it Yourself Fully Managed MySQL MySQL Oracle Oracle SQL Server SQL Server MariaDB Postgres …
  • 15. SQL NoSQL Do-it Yourself Fully Managed MongoDB DynamoDB Cassandra ElastiCache Redis SimpleDB Memcache
  • 16. Should I use Should I use SQL MySQL on EC2 or or NoSQL? RDS? Should I use MongoDB, ? Should I use Redis, Cassandra, or Memcache, or DynamoDB? ElastiCache?
  • 17. What are my What are my scale transactional and and latency needs? consistency needs? What are my ? What are my time to read/write, storage market and server and IOPS needs? control needs?
  • 18. Factors SQL NoSQL Application • App with complex business logic? • Web app with lots of users? Transactions • Complex txns, joins, updates? • Simple data model, updates, queries? Scale • Developer managed • Automatic, on-demand scaling Performance • Developer architected • Consistent, high performance at scale Availability • Architected for fail-over • Seamless and transparent Core Skills • SQL + Java/Ruby/Python/PhP • NoSQL + Java/Ruby/Python/PhP Best of both worlds: Possible to Use SQL and NoSQL models in one App
  • 19. Factors Do it Yourself (DIY) Fully Managed Replication • Granular, app managed • Transparent and configured Monitoring • Specific agents and custom • Automated and API driven Security • Root access, custom configs • Hardened by the service Resources • Requires more DBA resources and time • Requires less DBA resources and time Time to market • Sophistication vs. speed • Rapid iteration Core Skills • Systems, databases, monitoring • Applications, User focused Best of both worlds: Possible to manage different tiers differently
  • 20. Amazon RDS is a fully managed SQL database service. Choice of Database engines Simple to deploy and scale Reliable and cost effective Without any operational burden.
  • 21. Migration Backup and recovery Schema design Patching Query construction Configuration Query optimization Software upgrades Storage upgrades Frequent server upgrades Focus on the “innovation” Hardware crash Off load the “administration”
  • 22.  Multiple databases per instance  Standard user accounts  Connect and query using common MySQL tools & drivers  Tune engine parameters  Import and export data using standard MySQL tools (mysqldump)  Diagnostics  Native MySQL replication  SSL for encryption over the wire  Monitor metrics  Shell, super user or direct file system access (Think security!)
  • 23. ElastiCache is a fully managed Memcache caching service. Easy to set up and operate Scale cache clusters with push button ease Ultra fast response time for read scaling Without any operational burden.
  • 24. Amazon DynamoDB is a fully managed NoSQL database service. Store and retrieve any amount of data Scale throughput to millions of IO Single digit millisecond latencies Without any operational burden.
  • 25. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem “Select”, “insert”, “update” DescribeTable items Manage tables DeleteItem ListTables BatchGetItem Query Bulk select or update Query specific items OR Scan BatchWriteItem (max 1MB) scan the full table
  • 26. So, what are the tips and techniques for successful deployments?
  • 27. Educates millions of students Amazon EC2 Amazon DynamoDB Amazon Reaches millions of citizens Elasticache Amazon RDS Amazon Analyzes billions of Ads S3
  • 28. Kimo Educates millions of students Rosenbaum Reaches millions of citizens Analyzes billions of Ads
  • 29. Kimo Rosenbaum – Data Architect, Edmodo
  • 30. Where learning happens. Kimo Rosenbaum kimo@edmodo.com AWS re: Invent 2012
  • 31. Learning 101 • Largest, fastest growing social platform for education • Secure learning network for teachers and students • Browser, iOS, Android • Free for teachers and students
  • 32. Stats 101 • 100,000 schools • 14 million users • 7 million new users in the last year • 1 million visits daily
  • 33. Web Instance Auto scaling Group Amazon CloudWatch Amazon Route 53 Elastic Load Balancer Cache Cache Instance Instance Amazon Cloudfront Instances Amazon S3 RDS DB Instance RDS DB Instance RDS DB Instance Read Replica Read Replica Read Replica Availability Zone RDS DB Instance RDS DB Instance RDS DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance
  • 34. DBA 101 • Restore from snapshot • Replica creation • Parameter tuning • Metrics collection • Know your app/data
  • 35. Educates millions of students Jay Reaches millions of citizens Edwards Analyzes billions of Ads
  • 36. Jay Edwards – Database Engineer, Obama Campaign
  • 37. Me. • Twitter: First dedicated DBA • OFA: Lead Database Engineer • PalominoDB: CTO & VP/Operations
  • 38. Obama for America. • Technically sophisticated for a campaign • Not “web-scale” • Hockey-stick++ growth • Downtime hurts. A lot…really, really, really a lot.
  • 40. OFA Architecture RDS Read Replica ElastiCache RDS with DynamoDB Multi-AZ ELB
  • 41. Problems! • You always need more databases • OFA had 24+ schemas & 100+ RDS instances • You never have enough DBAs • OFA had 1 – 2 x 0.5 fulltime MySQL DBAs
  • 42. Why RDS? • Makes operational issues very easy • Need more replicas? BAM! • Upsize hardware? KAPOW! • Point in time restore? BIF!
  • 43. Why not RDS? • Hardware cap (vertical v. horizontal) • Sophisticated use-cases • Frequent topology changes • Multi-region replication (on their roadmap) • DBAs need busy work
  • 44. Educates millions of students Reaches millions of citizens Andy Analyzes billions of Ads Skalet
  • 45. Andy Skalet - CTO, BrandVerity
  • 48. Big Data? Cast your problem
  • 50. Case Study: Crawl history
  • 53. • Managed services let you focus on creating value • Amazon S3 - Very robust, handles large items, but you filter • Amazon DynamoDB - Extremely fast, scalable, good value • Must cast your problem as kvs or key + range • Amazon RDS - MySQL, without the headaches • Amazon ElastiCache - As memcached, fast kvs for small data • Multi column queries on big data? • Looking forward to the AWS solution
  • 54. Thank you Free aws.amazon.com/dynamodb Tier aws.amazon.com/rds aws.amazon.com/elasticache raghavas@amazon
  • 55. We are sincerely eager to hear your feedback on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance.