SlideShare una empresa de Scribd logo
1 de 24
Descargar para leer sin conexión
Webinar: Scaling MySQL
Benefits of Automatic Data Distribution
                                     December 13, 2012
Agenda


       1. Who We Are

       2. The Scalability Problem

       3. Benefits of Automatic Data Distribution

       4. Customer ROI/Case Studies

       5. Q & A
          (please type questions directly into the GoToWebinar side panel)




2
Who We Are

    Presenters:                                     Paul Campaniello,
                                                  VP of Global Marketing
                                              25 year technology veteran with
                                              marketing experience at Mendix,
                                              Lumigent, Savantis and Precise.




                Doron Levari, Founder
            A technologist and long-time
          veteran of the database industry.
         Prior to founding ScaleBase, Doron
                  was CEO to Aluna.


3
Pain Points – The Scalability Problem

• Thousands of new online and mobile
  apps launching every day
• Demand climbs for these apps and
  databases can’t keep up
• App must provide uninterrupted
  access and availability
• Database performance and
  scalability is critical




4
Big Data = Big Scaling Needs

       Big Data = Transactions + Interactions + Observations
               Sensors/RFID/Devices      Mobile Web       User Generated Content        Spatial & GPS Coordinates




                                                                                                                            BIG DATA
Petabytes      User Click Stream         Sentiment        Social Interactions & Feeds


               Web Logs               Dynamic Pricing       Search Marketing




                                                                                                 WEB
               Offer History          A/B Testing           Affiliate Networks
Terabytes                                                                                                 External
                                                                                                          Demographics
               Segmentation           Customer Touches




                                                                                 CRM
                                                                                                          Business Data
               Offer Details          Support Contacts                                                    Feeds


Gigabytes
                                                                                                  HD Video, Audio, Images
                                                                                   Behavioral
                                                    ERP


                    Purchase Detail
                                                                                   Targeting      Speech to Text
                    Purchase Record
                                                                                                  Product/Service Logs
                    Payment Record                                                 Dynamic
                                                                                   Funnels
                                                                                                  SMS/MMS
Megabytes



                                      Increasing Data Variety and Complexity

   5
                                           The 451 Group & Teradata
Scalability Pain



Infrastructure
Cost $
                   Large                     You just lost
                   Capital                    customers
                 Expenditure


                                                         Predicted
                                                         Demand

                               Opportunity                   Traditional
                                 Cost                        Hardware

                                                             Actual
                                                             Demand

                                                         Dynamic
                                                         Scaling


                                                                      time


    6
Ongoing “Scaling MySQL” Series

    • August 16 & September 20, 2012
       – Scaling MySQL: ScaleUp versus Scale Out

    • October 23, 2012
       – Methods and challenges to Scale out MySQL

    • Today
       – Benefits of Automatic Data Distribution

    • January 17, 2013
       – Catch 22 of read-write splitting



7
The Database Engine is the Bottleneck...

    • Every write operation is At Least 4 write operations inside the DB:
       – Data segment
       – Index segment
       – Undo segment
       – Transaction log
    • And Multiple Activities in the DB engine memory:
       – Buffer management
       – Locking
       – Thread locks/semaphores
       – Recovery tasks




8
The Database Engine is the Bottleneck

    • Every write operation is At Least 4 write operations inside the DB:
       – Data segment
       – Index segment
       – Undo segment                           Now multiply
       – Transaction log                           by 10TB
                                                 accessed by
    • And Multiple Activities in the DB engine memory:
                                                    10000
       – Buffer management
                                                 concurrent
       – Locking
                                                   sessions
       – Thread locks/semaphores
       – Recovery tasks




9
COI – Customer, Order, Item
           CUSTOMER                         ORDER                       ORDER_ITEM                    ITEM
C_ID     NAME      LOCATION   RANK   O_ID   C_ID   DATE         OI_ID    O_ID   QUANT   I_ID   I_ID   NAME
1        John      MA         10     1      1      2012-02-01   1        1      3       1      1      iPhone
2        James     AL         9      2      1      2012-02-01   2        1      6       2      2      iPad
3        Peter     CA         10     3      2      2012-02-01   3        2      4       1      3      iPad Mini
4        Chris     FL         8      4      6      2012-02-01   4        2      2       2      4      Kindle
5        Oliver    MA         9      5      6      2012-02-01   5        2      1       5      5      Kindle Fire
6        Allan     MA         9      6      8      2012-02-01   6        3      1       1      6      Galaxy S3
7        Janette   CA         8                                 7        3      6       5
8        David     MD         10                                8        4      8       3
                                                                9        4      9       4
                                                                10       5      2       6
                                                                11       6      1       5




    10
Requirements

 • Every day:
 • Updates                         Throughput

     – 30,000 new customers
     – 1,000,000 new orders, average of 5 items per order
     – Items catalog is updated once a day, nightly, on 11pm


                                    Latency
 • Queries
     – Top customers, rank 9 and up)
     – New orders, joins across the board…




11
Splitting the data

 • CUSTOMER – random (hash)
 • ORDER – derivative (C_ID)
 • ORDER_ITEM – transitive (O_ID -> C_ID)
 • ITEM – global table




12
Sliced Database
           CUSTOMER                         ORDER                       ORDER_ITEM                    ITEM
C_ID     NAME      LOCATION   RANK   O_ID   C_ID   DATE         OI_ID    O_ID   QUANT   I_ID   I_ID   NAME
1        John      MA         10     1      1      2012-02-01   1        1      3       1      1      iPhone
4        Chris     FL         8      2      1      2012-02-01   2        1      6       2      …      …
7        Janette   CA         8                                 3        2      4       1      6      Galaxy S3
                                                                4        2      2       2

                                                DB - 1          5        2      1       5


C_ID     NAME      LOCATION   RANK   O_ID   C_ID   DATE         OI_ID    O_ID   QUANT   I_ID   I_ID   NAME
2        James     AL         9      3      2      2012-02-01   6        3      1       1      1      iPhone
5        Oliver    MA         9      6      8      2012-02-01   7        3      6       5      …      …
8        David     MD         10                                11       6      1       5      6      Galaxy S3

                                                DB - 2

C_ID     NAME      LOCATION   RANK   O_ID   C_ID   DATE         OI_ID    O_ID   QUANT   I_ID   I_ID   NAME
3        Peter     CA         10     4      6      2012-02-01   8        4      8       3      1      iPhone
6        Allan     MA         9      5      6      2012-02-01   9        4      9       4      …      …
                                                                10       5      2       6      6      Galaxy S3
                                                DB - 3
    13
Requirements
                                                               Distribution
 • Every day:
 • Updates                         Throughput

     – 30,000 new customers
     – 1,000,000 new orders, average of 5 items per order
     – Items catalog is updated once a day, nightly, on 11pm

                                                               Parallelism
                                    Latency
 • Queries
     – Top customers, rank 9 and up)
     – New orders, joins across the board…




14
Automatic Data Distribution

 • The ultimate way to scale
 • Provides significant performance improvements
 • The only way to really improve read and also writes
 • Good for scaling high session-volume reads and writes
 • Good for scaling high data-volume reads and writes
 • Home-grown implementations have drawbacks




15
Scale Out Features and Benefits

     Feature                                 Benefit
     Parallel query execution                Great performance of cross-db queries &
                                             maintenance commands
     Query result aggregation                Support of sophisticated cross-db queries, even with
                                             ORDER BY, GROUP BY, LIMIT, Aggregate functions…

     Online data redistribution              Flexibility: no need to over-provision
                                             No downtime

     100% compatible MySQL proxy             Applications unmodified
                                             Standard MySQL tools and interfaces
     MySQL databases untouched               Data is safe within MySQL InnoDB/MyISAM/any

     Data distribution review and analysis   Optimization of data distribution policy

     Data consistency verifier               Validate system-wide data consistency

     Real-time monitoring and alerts         Simplify management, reduce TCO




16
Scale Out Provides Immediate & Tangible Value



     Application Server            Database A    Standby A




     Application Server           Database B     Standby B




                                  Database C    Standby C
            BI




                                 Database D     Standby D
       Management

17
Typical Scale Out (ScaleBase) Deployment



     Application Server                           Database A    Standby A

                              ScaleBase
                          Central Management




     Application Server                           Database B    Standby B



                               ScaleBase
                          Data Traffic Manager


                                                 Database C    Standby C
            BI




                                                 Database D    Standby D
       Management

18
Choose Your Scale-out Path


                              Data Distribution


           Database Size



                                      Read/Write Splitting




                           1 DB?
                           Good for me!




                               # of concurrent sessions
19
Scaling Out Achieves Unlimited Scalability

             160000

             140000

             120000

             100000
Throughput




                                                                                               84000
             80000                                                                                     Throughput (TPM)
                                                                                                       Total DB Size (MB)
             60000                                                                60000                # Connections
                                                                     48000
             40000
                                                        36000
                                              24000                                            2500
             20000                                                                2000
                                     12000              1500         1500
                          6000                1000
                 0        500        500
                      1          2           4        6          8           10           14
                                              Number of Databases

     20
Detailed Scale Out Case Studies




     Nokia               AppDynamics             Mozilla           Solar Edge
     • Device Apps App   • Next gen APM          • New Product/    • Next Gen
     • Availability        company                 Next Gen App/     Monitoring App
     • Scalability       • Scalability for the     AppStore        • Massive Scale
     • Geo-clustering      Netflix               • Scalability     • Monitors real
                           implementation        • Geo-sharding      time data from
     • 100 Apps
                                                                     thousands of
     • 300 MySQL DB
                                                                     distributed
                                                                     systems




21
Summary

     • Database scalability is a significant problem
         – App explosion, Big Data, Mobile
     • Scale Up helps somewhat, but Scale Out provides
       a long-term, cost-effective solution

     • ScaleBase has an effective Scale Out
       solution with a proven ROI
         – Improves performance &
           requires NO changes to
           your existing infrastructure
     • Choose your scale-out path....
         – The ScaleBase platform enables
           you to start with R/W splitting and
           grow into automatic data distribution

22
Questions (please enter directly into the GTW side panel)



617.630.2800

www.ScaleBase.com

doron.levari@scalebase.com

paul.campaniello@scalebase.com


23
Thank You
24

Más contenido relacionado

La actualidad más candente

Sql server 2012 smart dive presentation 20120126
Sql server 2012 smart dive presentation 20120126Sql server 2012 smart dive presentation 20120126
Sql server 2012 smart dive presentation 20120126Andrew Mauch
 
Is pervasive governance_part_of_your_ecm_strategy
Is pervasive governance_part_of_your_ecm_strategyIs pervasive governance_part_of_your_ecm_strategy
Is pervasive governance_part_of_your_ecm_strategyQuestexConf
 
Make Your Business More Flexible with Scalable Business Process Management So...
Make Your Business More Flexible with Scalable Business Process Management So...Make Your Business More Flexible with Scalable Business Process Management So...
Make Your Business More Flexible with Scalable Business Process Management So...Perficient, Inc.
 
Smart Micro Credit Business Suite
Smart Micro Credit Business SuiteSmart Micro Credit Business Suite
Smart Micro Credit Business SuiteFahad Iftikhar
 
Supply chain presentation 11 2006
Supply chain presentation 11 2006Supply chain presentation 11 2006
Supply chain presentation 11 2006Girard Brewer
 
Mike Stolz Dramatic Scalability
Mike Stolz Dramatic ScalabilityMike Stolz Dramatic Scalability
Mike Stolz Dramatic Scalabilitydeimos
 
Big Data i CSC's optik, CSC Representative
Big Data i CSC's optik, CSC RepresentativeBig Data i CSC's optik, CSC Representative
Big Data i CSC's optik, CSC RepresentativeIBM Danmark
 
Ariba E-Invoicing - Strategies for Collaborative Commerce
Ariba E-Invoicing - Strategies for Collaborative CommerceAriba E-Invoicing - Strategies for Collaborative Commerce
Ariba E-Invoicing - Strategies for Collaborative CommerceSAP Ariba
 
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...CradlePoint
 
Testing and Optimization
Testing and OptimizationTesting and Optimization
Testing and OptimizationDatalicious
 
Unilog 2011 core services sanitized
Unilog 2011 core services   sanitizedUnilog 2011 core services   sanitized
Unilog 2011 core services sanitizedbasuchit
 
Supplier Collaboration - Doing it the right way
Supplier Collaboration - Doing it the right waySupplier Collaboration - Doing it the right way
Supplier Collaboration - Doing it the right wayManthan
 
STRATEGIC USE OF MIS DRGORAD
STRATEGIC USE OF MIS DRGORADSTRATEGIC USE OF MIS DRGORAD
STRATEGIC USE OF MIS DRGORADDeepak R Gorad
 
Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Cana Ko
 
TH e-GIF on SOA Using Open Enterprise Architecture
TH e-GIF on SOA Using Open Enterprise ArchitectureTH e-GIF on SOA Using Open Enterprise Architecture
TH e-GIF on SOA Using Open Enterprise ArchitectureThanachart Numnonda
 
Elastic caching for scalability, dynamic growth and performance
Elastic caching for scalability, dynamic growth and performanceElastic caching for scalability, dynamic growth and performance
Elastic caching for scalability, dynamic growth and performancecathylums
 
Increase Agility & ROI: BPM in Business Support Systems
Increase Agility & ROI: BPM in Business Support SystemsIncrease Agility & ROI: BPM in Business Support Systems
Increase Agility & ROI: BPM in Business Support SystemsSrikanth Minnam
 
Managing Unprecedented Change with Business Transformation
Managing Unprecedented Change with Business TransformationManaging Unprecedented Change with Business Transformation
Managing Unprecedented Change with Business TransformationCisco Canada
 

La actualidad más candente (20)

Sql server 2012 smart dive presentation 20120126
Sql server 2012 smart dive presentation 20120126Sql server 2012 smart dive presentation 20120126
Sql server 2012 smart dive presentation 20120126
 
Is pervasive governance_part_of_your_ecm_strategy
Is pervasive governance_part_of_your_ecm_strategyIs pervasive governance_part_of_your_ecm_strategy
Is pervasive governance_part_of_your_ecm_strategy
 
Make Your Business More Flexible with Scalable Business Process Management So...
Make Your Business More Flexible with Scalable Business Process Management So...Make Your Business More Flexible with Scalable Business Process Management So...
Make Your Business More Flexible with Scalable Business Process Management So...
 
Smart Micro Credit Business Suite
Smart Micro Credit Business SuiteSmart Micro Credit Business Suite
Smart Micro Credit Business Suite
 
Supply chain presentation 11 2006
Supply chain presentation 11 2006Supply chain presentation 11 2006
Supply chain presentation 11 2006
 
Mike Stolz Dramatic Scalability
Mike Stolz Dramatic ScalabilityMike Stolz Dramatic Scalability
Mike Stolz Dramatic Scalability
 
Big Data i CSC's optik, CSC Representative
Big Data i CSC's optik, CSC RepresentativeBig Data i CSC's optik, CSC Representative
Big Data i CSC's optik, CSC Representative
 
Ariba E-Invoicing - Strategies for Collaborative Commerce
Ariba E-Invoicing - Strategies for Collaborative CommerceAriba E-Invoicing - Strategies for Collaborative Commerce
Ariba E-Invoicing - Strategies for Collaborative Commerce
 
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...
Speed to Deployment: Implement Instant and Pop-up Networks Using Flexible 3G/...
 
Testing and Optimization
Testing and OptimizationTesting and Optimization
Testing and Optimization
 
Unilog 2011 core services sanitized
Unilog 2011 core services   sanitizedUnilog 2011 core services   sanitized
Unilog 2011 core services sanitized
 
Supplier Collaboration - Doing it the right way
Supplier Collaboration - Doing it the right waySupplier Collaboration - Doing it the right way
Supplier Collaboration - Doing it the right way
 
Manthan
Manthan Manthan
Manthan
 
STRATEGIC USE OF MIS DRGORAD
STRATEGIC USE OF MIS DRGORADSTRATEGIC USE OF MIS DRGORAD
STRATEGIC USE OF MIS DRGORAD
 
Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831
 
TH e-GIF on SOA Using Open Enterprise Architecture
TH e-GIF on SOA Using Open Enterprise ArchitectureTH e-GIF on SOA Using Open Enterprise Architecture
TH e-GIF on SOA Using Open Enterprise Architecture
 
Elastic caching for scalability, dynamic growth and performance
Elastic caching for scalability, dynamic growth and performanceElastic caching for scalability, dynamic growth and performance
Elastic caching for scalability, dynamic growth and performance
 
Synergy Fact Sheets
Synergy Fact SheetsSynergy Fact Sheets
Synergy Fact Sheets
 
Increase Agility & ROI: BPM in Business Support Systems
Increase Agility & ROI: BPM in Business Support SystemsIncrease Agility & ROI: BPM in Business Support Systems
Increase Agility & ROI: BPM in Business Support Systems
 
Managing Unprecedented Change with Business Transformation
Managing Unprecedented Change with Business TransformationManaging Unprecedented Change with Business Transformation
Managing Unprecedented Change with Business Transformation
 

Destacado

ScaleBase Backs Mozilla's new app store
ScaleBase Backs Mozilla's new app storeScaleBase Backs Mozilla's new app store
ScaleBase Backs Mozilla's new app storeScaleBase
 
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase
 
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...ScaleBase
 
Database Scalability - The Shard Conflict
Database Scalability - The Shard ConflictDatabase Scalability - The Shard Conflict
Database Scalability - The Shard ConflictScaleBase
 
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...ScaleBase
 
Challenges in Querying a Distributed Relational Database
Challenges in Querying a Distributed Relational DatabaseChallenges in Querying a Distributed Relational Database
Challenges in Querying a Distributed Relational DatabaseScaleBase
 
ScaleBase Webinar: Strategies for scaling MySQL
ScaleBase Webinar: Strategies for scaling MySQLScaleBase Webinar: Strategies for scaling MySQL
ScaleBase Webinar: Strategies for scaling MySQLScaleBase
 
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...ScaleBase
 
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016Amazon Web Services Korea
 
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQL
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQLChoosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQL
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQLScaleBase
 

Destacado (10)

ScaleBase Backs Mozilla's new app store
ScaleBase Backs Mozilla's new app storeScaleBase Backs Mozilla's new app store
ScaleBase Backs Mozilla's new app store
 
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
 
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...
Distributed RDBMS: Data Distribution Policy: Part 1 - What is a Data Distribu...
 
Database Scalability - The Shard Conflict
Database Scalability - The Shard ConflictDatabase Scalability - The Shard Conflict
Database Scalability - The Shard Conflict
 
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...
Distributed RDBMS: Data Distribution Policy: Part 2 - Creating a Data Distrib...
 
Challenges in Querying a Distributed Relational Database
Challenges in Querying a Distributed Relational DatabaseChallenges in Querying a Distributed Relational Database
Challenges in Querying a Distributed Relational Database
 
ScaleBase Webinar: Strategies for scaling MySQL
ScaleBase Webinar: Strategies for scaling MySQLScaleBase Webinar: Strategies for scaling MySQL
ScaleBase Webinar: Strategies for scaling MySQL
 
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...
Distributed RDBMS: Data Distribution Policy: Part 3 - Changing Your Data Dist...
 
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016
AWS 클라우드로 천만명 웹 서비스 확장하기 - 윤석찬 백승현 - AWS Summit 2016
 
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQL
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQLChoosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQL
Choosing a Next Gen Database: the New World Order of NoSQL, NewSQL, and MySQL
 

Similar a Scaling MySQL: Benefits of Automatic Data Distribution

Unified big data architecture
Unified big data architectureUnified big data architecture
Unified big data architectureDataWorks Summit
 
Big Data, Hadoop, Hortonworks and Microsoft HDInsight
Big Data, Hadoop, Hortonworks and Microsoft HDInsightBig Data, Hadoop, Hortonworks and Microsoft HDInsight
Big Data, Hadoop, Hortonworks and Microsoft HDInsightHortonworks
 
Powering Next Generation Data Architecture With Apache Hadoop
Powering Next Generation Data Architecture With Apache HadoopPowering Next Generation Data Architecture With Apache Hadoop
Powering Next Generation Data Architecture With Apache HadoopHortonworks
 
The Comprehensive Approach: A Unified Information Architecture
The Comprehensive Approach: A Unified Information ArchitectureThe Comprehensive Approach: A Unified Information Architecture
The Comprehensive Approach: A Unified Information ArchitectureInside Analysis
 
Asug SAP HANA Presentation - Perceptive Technologies SAP
Asug SAP HANA Presentation - Perceptive Technologies SAPAsug SAP HANA Presentation - Perceptive Technologies SAP
Asug SAP HANA Presentation - Perceptive Technologies SAPBrendan Kane
 
Talend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data PlatformTalend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data PlatformHortonworks
 
Integrating social media monitoring, analytics and engagment marshall sponde...
Integrating social media monitoring, analytics and engagment  marshall sponde...Integrating social media monitoring, analytics and engagment  marshall sponde...
Integrating social media monitoring, analytics and engagment marshall sponde...Marshall Sponder
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshowAccenture
 
Informatica World 2006 - MDM Data Quality
Informatica World 2006 - MDM Data QualityInformatica World 2006 - MDM Data Quality
Informatica World 2006 - MDM Data QualityDatabase Architechs
 
The Next Generation of Big Data Analytics
The Next Generation of Big Data AnalyticsThe Next Generation of Big Data Analytics
The Next Generation of Big Data AnalyticsHortonworks
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architecturepcherukumalla
 
Data Warehouse Architecture
Data Warehouse ArchitectureData Warehouse Architecture
Data Warehouse Architecturepcherukumalla
 
BI Self-Service Keys to Success and QlikView Overview
BI Self-Service Keys to Success and QlikView OverviewBI Self-Service Keys to Success and QlikView Overview
BI Self-Service Keys to Success and QlikView OverviewSenturus
 
OpTier McKinsey Big Data Overview
OpTier McKinsey Big Data OverviewOpTier McKinsey Big Data Overview
OpTier McKinsey Big Data Overviewnickychu
 
McKinsey Big Data Overview
McKinsey Big Data OverviewMcKinsey Big Data Overview
McKinsey Big Data Overviewoptier
 
Silicon Halton Meetup 41 - post event deck
Silicon Halton Meetup 41 - post event deckSilicon Halton Meetup 41 - post event deck
Silicon Halton Meetup 41 - post event deckSilicon Halton
 
McKinsey Big Data Overview
McKinsey Big Data OverviewMcKinsey Big Data Overview
McKinsey Big Data Overviewoptier
 
2012數位趨勢高峰論壇--功典集團執行長陳傑豪
2012數位趨勢高峰論壇--功典集團執行長陳傑豪2012數位趨勢高峰論壇--功典集團執行長陳傑豪
2012數位趨勢高峰論壇--功典集團執行長陳傑豪Neo Marketing Workshop
 

Similar a Scaling MySQL: Benefits of Automatic Data Distribution (20)

vBACD July 2012 - Apache Hadoop, Now and Beyond
vBACD July 2012 - Apache Hadoop, Now and BeyondvBACD July 2012 - Apache Hadoop, Now and Beyond
vBACD July 2012 - Apache Hadoop, Now and Beyond
 
Unified big data architecture
Unified big data architectureUnified big data architecture
Unified big data architecture
 
Big Data, Hadoop, Hortonworks and Microsoft HDInsight
Big Data, Hadoop, Hortonworks and Microsoft HDInsightBig Data, Hadoop, Hortonworks and Microsoft HDInsight
Big Data, Hadoop, Hortonworks and Microsoft HDInsight
 
2012 06 hortonworks paris hug
2012 06 hortonworks paris hug2012 06 hortonworks paris hug
2012 06 hortonworks paris hug
 
Powering Next Generation Data Architecture With Apache Hadoop
Powering Next Generation Data Architecture With Apache HadoopPowering Next Generation Data Architecture With Apache Hadoop
Powering Next Generation Data Architecture With Apache Hadoop
 
The Comprehensive Approach: A Unified Information Architecture
The Comprehensive Approach: A Unified Information ArchitectureThe Comprehensive Approach: A Unified Information Architecture
The Comprehensive Approach: A Unified Information Architecture
 
Asug SAP HANA Presentation - Perceptive Technologies SAP
Asug SAP HANA Presentation - Perceptive Technologies SAPAsug SAP HANA Presentation - Perceptive Technologies SAP
Asug SAP HANA Presentation - Perceptive Technologies SAP
 
Talend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data PlatformTalend Open Studio and Hortonworks Data Platform
Talend Open Studio and Hortonworks Data Platform
 
Integrating social media monitoring, analytics and engagment marshall sponde...
Integrating social media monitoring, analytics and engagment  marshall sponde...Integrating social media monitoring, analytics and engagment  marshall sponde...
Integrating social media monitoring, analytics and engagment marshall sponde...
 
Hortonworks roadshow
Hortonworks roadshowHortonworks roadshow
Hortonworks roadshow
 
Informatica World 2006 - MDM Data Quality
Informatica World 2006 - MDM Data QualityInformatica World 2006 - MDM Data Quality
Informatica World 2006 - MDM Data Quality
 
The Next Generation of Big Data Analytics
The Next Generation of Big Data AnalyticsThe Next Generation of Big Data Analytics
The Next Generation of Big Data Analytics
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architecture
 
Data Warehouse Architecture
Data Warehouse ArchitectureData Warehouse Architecture
Data Warehouse Architecture
 
BI Self-Service Keys to Success and QlikView Overview
BI Self-Service Keys to Success and QlikView OverviewBI Self-Service Keys to Success and QlikView Overview
BI Self-Service Keys to Success and QlikView Overview
 
OpTier McKinsey Big Data Overview
OpTier McKinsey Big Data OverviewOpTier McKinsey Big Data Overview
OpTier McKinsey Big Data Overview
 
McKinsey Big Data Overview
McKinsey Big Data OverviewMcKinsey Big Data Overview
McKinsey Big Data Overview
 
Silicon Halton Meetup 41 - post event deck
Silicon Halton Meetup 41 - post event deckSilicon Halton Meetup 41 - post event deck
Silicon Halton Meetup 41 - post event deck
 
McKinsey Big Data Overview
McKinsey Big Data OverviewMcKinsey Big Data Overview
McKinsey Big Data Overview
 
2012數位趨勢高峰論壇--功典集團執行長陳傑豪
2012數位趨勢高峰論壇--功典集團執行長陳傑豪2012數位趨勢高峰論壇--功典集團執行長陳傑豪
2012數位趨勢高峰論壇--功典集團執行長陳傑豪
 

Scaling MySQL: Benefits of Automatic Data Distribution

  • 1. Webinar: Scaling MySQL Benefits of Automatic Data Distribution December 13, 2012
  • 2. Agenda 1. Who We Are 2. The Scalability Problem 3. Benefits of Automatic Data Distribution 4. Customer ROI/Case Studies 5. Q & A (please type questions directly into the GoToWebinar side panel) 2
  • 3. Who We Are Presenters: Paul Campaniello, VP of Global Marketing 25 year technology veteran with marketing experience at Mendix, Lumigent, Savantis and Precise. Doron Levari, Founder A technologist and long-time veteran of the database industry. Prior to founding ScaleBase, Doron was CEO to Aluna. 3
  • 4. Pain Points – The Scalability Problem • Thousands of new online and mobile apps launching every day • Demand climbs for these apps and databases can’t keep up • App must provide uninterrupted access and availability • Database performance and scalability is critical 4
  • 5. Big Data = Big Scaling Needs Big Data = Transactions + Interactions + Observations Sensors/RFID/Devices Mobile Web User Generated Content Spatial & GPS Coordinates BIG DATA Petabytes User Click Stream Sentiment Social Interactions & Feeds Web Logs Dynamic Pricing Search Marketing WEB Offer History A/B Testing Affiliate Networks Terabytes External Demographics Segmentation Customer Touches CRM Business Data Offer Details Support Contacts Feeds Gigabytes HD Video, Audio, Images Behavioral ERP Purchase Detail Targeting Speech to Text Purchase Record Product/Service Logs Payment Record Dynamic Funnels SMS/MMS Megabytes Increasing Data Variety and Complexity 5 The 451 Group & Teradata
  • 6. Scalability Pain Infrastructure Cost $ Large You just lost Capital customers Expenditure Predicted Demand Opportunity Traditional Cost Hardware Actual Demand Dynamic Scaling time 6
  • 7. Ongoing “Scaling MySQL” Series • August 16 & September 20, 2012 – Scaling MySQL: ScaleUp versus Scale Out • October 23, 2012 – Methods and challenges to Scale out MySQL • Today – Benefits of Automatic Data Distribution • January 17, 2013 – Catch 22 of read-write splitting 7
  • 8. The Database Engine is the Bottleneck... • Every write operation is At Least 4 write operations inside the DB: – Data segment – Index segment – Undo segment – Transaction log • And Multiple Activities in the DB engine memory: – Buffer management – Locking – Thread locks/semaphores – Recovery tasks 8
  • 9. The Database Engine is the Bottleneck • Every write operation is At Least 4 write operations inside the DB: – Data segment – Index segment – Undo segment Now multiply – Transaction log by 10TB accessed by • And Multiple Activities in the DB engine memory: 10000 – Buffer management concurrent – Locking sessions – Thread locks/semaphores – Recovery tasks 9
  • 10. COI – Customer, Order, Item CUSTOMER ORDER ORDER_ITEM ITEM C_ID NAME LOCATION RANK O_ID C_ID DATE OI_ID O_ID QUANT I_ID I_ID NAME 1 John MA 10 1 1 2012-02-01 1 1 3 1 1 iPhone 2 James AL 9 2 1 2012-02-01 2 1 6 2 2 iPad 3 Peter CA 10 3 2 2012-02-01 3 2 4 1 3 iPad Mini 4 Chris FL 8 4 6 2012-02-01 4 2 2 2 4 Kindle 5 Oliver MA 9 5 6 2012-02-01 5 2 1 5 5 Kindle Fire 6 Allan MA 9 6 8 2012-02-01 6 3 1 1 6 Galaxy S3 7 Janette CA 8 7 3 6 5 8 David MD 10 8 4 8 3 9 4 9 4 10 5 2 6 11 6 1 5 10
  • 11. Requirements • Every day: • Updates Throughput – 30,000 new customers – 1,000,000 new orders, average of 5 items per order – Items catalog is updated once a day, nightly, on 11pm Latency • Queries – Top customers, rank 9 and up) – New orders, joins across the board… 11
  • 12. Splitting the data • CUSTOMER – random (hash) • ORDER – derivative (C_ID) • ORDER_ITEM – transitive (O_ID -> C_ID) • ITEM – global table 12
  • 13. Sliced Database CUSTOMER ORDER ORDER_ITEM ITEM C_ID NAME LOCATION RANK O_ID C_ID DATE OI_ID O_ID QUANT I_ID I_ID NAME 1 John MA 10 1 1 2012-02-01 1 1 3 1 1 iPhone 4 Chris FL 8 2 1 2012-02-01 2 1 6 2 … … 7 Janette CA 8 3 2 4 1 6 Galaxy S3 4 2 2 2 DB - 1 5 2 1 5 C_ID NAME LOCATION RANK O_ID C_ID DATE OI_ID O_ID QUANT I_ID I_ID NAME 2 James AL 9 3 2 2012-02-01 6 3 1 1 1 iPhone 5 Oliver MA 9 6 8 2012-02-01 7 3 6 5 … … 8 David MD 10 11 6 1 5 6 Galaxy S3 DB - 2 C_ID NAME LOCATION RANK O_ID C_ID DATE OI_ID O_ID QUANT I_ID I_ID NAME 3 Peter CA 10 4 6 2012-02-01 8 4 8 3 1 iPhone 6 Allan MA 9 5 6 2012-02-01 9 4 9 4 … … 10 5 2 6 6 Galaxy S3 DB - 3 13
  • 14. Requirements Distribution • Every day: • Updates Throughput – 30,000 new customers – 1,000,000 new orders, average of 5 items per order – Items catalog is updated once a day, nightly, on 11pm Parallelism Latency • Queries – Top customers, rank 9 and up) – New orders, joins across the board… 14
  • 15. Automatic Data Distribution • The ultimate way to scale • Provides significant performance improvements • The only way to really improve read and also writes • Good for scaling high session-volume reads and writes • Good for scaling high data-volume reads and writes • Home-grown implementations have drawbacks 15
  • 16. Scale Out Features and Benefits Feature Benefit Parallel query execution Great performance of cross-db queries & maintenance commands Query result aggregation Support of sophisticated cross-db queries, even with ORDER BY, GROUP BY, LIMIT, Aggregate functions… Online data redistribution Flexibility: no need to over-provision No downtime 100% compatible MySQL proxy Applications unmodified Standard MySQL tools and interfaces MySQL databases untouched Data is safe within MySQL InnoDB/MyISAM/any Data distribution review and analysis Optimization of data distribution policy Data consistency verifier Validate system-wide data consistency Real-time monitoring and alerts Simplify management, reduce TCO 16
  • 17. Scale Out Provides Immediate & Tangible Value Application Server Database A Standby A Application Server Database B Standby B Database C Standby C BI Database D Standby D Management 17
  • 18. Typical Scale Out (ScaleBase) Deployment Application Server Database A Standby A ScaleBase Central Management Application Server Database B Standby B ScaleBase Data Traffic Manager Database C Standby C BI Database D Standby D Management 18
  • 19. Choose Your Scale-out Path Data Distribution Database Size Read/Write Splitting 1 DB? Good for me! # of concurrent sessions 19
  • 20. Scaling Out Achieves Unlimited Scalability 160000 140000 120000 100000 Throughput 84000 80000 Throughput (TPM) Total DB Size (MB) 60000 60000 # Connections 48000 40000 36000 24000 2500 20000 2000 12000 1500 1500 6000 1000 0 500 500 1 2 4 6 8 10 14 Number of Databases 20
  • 21. Detailed Scale Out Case Studies Nokia AppDynamics Mozilla Solar Edge • Device Apps App • Next gen APM • New Product/ • Next Gen • Availability company Next Gen App/ Monitoring App • Scalability • Scalability for the AppStore • Massive Scale • Geo-clustering Netflix • Scalability • Monitors real implementation • Geo-sharding time data from • 100 Apps thousands of • 300 MySQL DB distributed systems 21
  • 22. Summary • Database scalability is a significant problem – App explosion, Big Data, Mobile • Scale Up helps somewhat, but Scale Out provides a long-term, cost-effective solution • ScaleBase has an effective Scale Out solution with a proven ROI – Improves performance & requires NO changes to your existing infrastructure • Choose your scale-out path.... – The ScaleBase platform enables you to start with R/W splitting and grow into automatic data distribution 22
  • 23. Questions (please enter directly into the GTW side panel) 617.630.2800 www.ScaleBase.com doron.levari@scalebase.com paul.campaniello@scalebase.com 23