SlideShare una empresa de Scribd logo
1 de 12
Dynamic Consistency


                                           April 2013
                                       Cloudian KK & Inc.



(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
                                                              1
Purpose of this document

           Cloudian developed “Dynamic Consistency” and released it in
           Cloudian version 2.4.
           “Dynamic Consistency” is a unique technology that dynamically
           changes data consistency in a high-availability configuration
           Provides the following benefits:
               - Continue services in case of failures without IT intervention
               - Maintain highest possible data consistency
           This document covers the following topics:
                What is data consistency in a distributed system?
                What is the problem with static data consistency?
                How does “Dynamic Consistency” solve the problem?



                                                                                 2
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
What is “Cloudian”?

           Cloudian is packaged software product that provides;
              Distributed object storage system
              Amazon S3 compatible API
              Unlimited scalability to hundreds of petabytes
              Data replications in multi-data centers
              Tunable data consistency




           Cloudian is being used by large organizations to offer cloud storage
           services/systems



                                                                            3
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
How Replication works in the distributed system?


        One object is replicated to                              Even in the case of a
        multiple servers                                         server failure, client can
                                                                 get the object

                        Client                                             Client

Put “object”                                                                         Get “object”




     Server            Server             Server              Server      Server       Server
       1                 2                  3                   1           2            3




                   Replication

                                                                                          4
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
Cloudian “Data Consistency” levels are tunable
     Write Consistency




     Read Consistency




                                                              5
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
How “Data Consistency” works in the
     distributed system?
        After “all” objects are                                After “some” objects are
        replicated, Client receives                            replicated, Client receives
        acknowledgement                                        acknowledgement

      Client             All example
                                                              Client       QUORUM(R+W>N) example
                         (N) Replications=3                                (N) Replications=3
                         (W) Write=3                                       (W) Write=2
                         (R) Read=any                                      (R) Read=2
            “ack”                                                 “ack”


     Server            Server             Server              Server      Server       Server
       1                 2                  3                   1           2            3




                                       Client                                        Client

           Data is always consistent                            Data is consistent by
                                                                                             6
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
                                                                QUORUM
Problem: “All” is disruptive in case of failure &
     slow for replication over slow links
          “All” or “QUORUM” cannot be completed in the case of failure
          “All” or “QUORUM” is slower for clients/applications

Assumptions:
• 3 Replications
• 2 in DC1                           Client
• 1 in DC2


                                      No “ack”



      Server           Server           Server                Server      Server       Server
        1                2                3                     4           5            6




                     Data Center 1                                     Data Center 2
                                                                                          7
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
Problem: “Local_QUORUM ” does not guarantee
     “Data Consistent” in the whole system

        “Local _Quorum” cannot provide “data consistency” in the whole
        system

Assumptions:                         Client                   Client       Client may get “object”
• 3 Replications                                                           before updated
• 2 in DC1
• 1 in DC2
                                          “ack”



      Server           Server           Server                 Server      Server       Server
        1                2                3                      4           5            6




                   Data Center 1                                        Data Center 2

                 Data is consistent only when object is read from DC1
                                                                                            8
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
“Cloudian” provides Dynamic & Flexible
     Consistency
                        Synchronous                                               Asynchronous
                         Replication                                               Replication
 Traditional
 Approach
  (static &
                    + No Data Loss                                          + Not dependent on
 inflexible)        - Failures if DR site                                     availability of DR site
                      not reachable                                         - Data Loss in case of
                                                                              Disaster

                        Synchronous           Each             Local   Quorum      Asynchronous
   Cloudian              Replication         Quorum           Quorum                Replication
   Multi-DC
   (static &
   flexible)



Cloudian 2.4            Synchronous           Each             Local   Quorum     Asynchronous
  Multi-DC             Replication #1        Quorum           Quorum              Replication #3
(dynamic &                                                      #2
  flexible)
                                                                                            9
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
Solution: “Dynamic Consistency” changes
     consistency level dynamically (1/2)

      1. System operates at “All” to keep “Data Consistency”

Assumptions:                         Client                   Client
• 3 Replications
• 2 in DC1
• 1 in DC2                                                                       Client always get
                                       “ack”                                     the same “object”




      Server           Server           Server                 Server      Server       Server
        1                2                3                      4           5            6




                     Data Center 1                                      Data Center 2

                          Data is consistent in the whole system
                                                                                           10
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
Solution: “Dynamic Consistency” changes
     consistency level dynamically (2/2)

      2. System automatically falls back to “Local_QUORUM” in case
         of data center failure
Assumptions:
• 3 Replications
• 2 in DC1                           Client                   Client
• 1 in DC2
                                                                       Client gets “object” from
                                       “ack”                           surviving data center



      Server           Server           Server                  Server       Server      Server
        1                2                3                       4            5           6




                  Data Center 1                                        Data Center 2

                                                                                            11
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
For more information:

       Contact by Email: info@cloudian.com
       Homepage:                         www.cloudian.com
                                         www.cloudian.jp
       Cloudian Community Edition:
                http://www.cloudian.com/get-started.html

       30⽇間無料トライル:
                 http://www.cloudian.jp/cloud-storage-products/cloudian_eval_agr.php

                                                                                  12
(c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.

Más contenido relacionado

La actualidad más candente

2020 07-30 elastic agent + ingest management
2020 07-30 elastic agent + ingest management2020 07-30 elastic agent + ingest management
2020 07-30 elastic agent + ingest managementDaliya Spasova
 
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트:: A...
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트::  A...실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트::  A...
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트:: A...Amazon Web Services Korea
 
Upgrading to VMware vSphere 6.0
Upgrading to VMware vSphere 6.0Upgrading to VMware vSphere 6.0
Upgrading to VMware vSphere 6.0Tim Carman
 
Az 104 session 5: Azure networking
Az 104 session 5: Azure networkingAz 104 session 5: Azure networking
Az 104 session 5: Azure networkingAzureEzy1
 
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017AWSKRUG - AWS한국사용자모임
 
Type of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleType of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleHimani Singh
 
Infrastructure Security by Sivamurthy Hiremath
Infrastructure Security by Sivamurthy HiremathInfrastructure Security by Sivamurthy Hiremath
Infrastructure Security by Sivamurthy HiremathClubHack
 
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech Talks
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech TalksAmazon GuardDuty - Let's Attack My Account! - AWS Online Tech Talks
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech TalksAmazon Web Services
 
データ活用を加速するAWS分析サービスのご紹介
データ活用を加速するAWS分析サービスのご紹介データ活用を加速するAWS分析サービスのご紹介
データ活用を加速するAWS分析サービスのご紹介Amazon Web Services Japan
 
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイント
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイントJPC2018[E3]NoOps で進める! Microsoft 365 提案のポイント
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイントMPN Japan
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEMJohn Hubbard
 
Cassandraのバックアップと運用を考える
Cassandraのバックアップと運用を考えるCassandraのバックアップと運用を考える
Cassandraのバックアップと運用を考えるKazutaka Tomita
 
Splunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk
 
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術Masaya Aoyama
 
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼Amazon Web Services Korea
 
AWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAmazon Web Services
 
Azure Networking: Innovative Features and Multi-VNet Topologies
Azure Networking: Innovative Features and Multi-VNet TopologiesAzure Networking: Innovative Features and Multi-VNet Topologies
Azure Networking: Innovative Features and Multi-VNet TopologiesMarius Zaharia
 
20200218 AWS Black Belt Online Seminar Next Generation Redshift
20200218 AWS Black Belt Online Seminar Next Generation Redshift20200218 AWS Black Belt Online Seminar Next Generation Redshift
20200218 AWS Black Belt Online Seminar Next Generation RedshiftAmazon Web Services Japan
 

La actualidad más candente (20)

2020 07-30 elastic agent + ingest management
2020 07-30 elastic agent + ingest management2020 07-30 elastic agent + ingest management
2020 07-30 elastic agent + ingest management
 
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트:: A...
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트::  A...실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트::  A...
실전! AWS 하이브리드 네트워킹 (AWS Direct Connect 및 VPN 데모 세션) - 강동환, AWS 솔루션즈 아키텍트:: A...
 
Upgrading to VMware vSphere 6.0
Upgrading to VMware vSphere 6.0Upgrading to VMware vSphere 6.0
Upgrading to VMware vSphere 6.0
 
Az 104 session 5: Azure networking
Az 104 session 5: Azure networkingAz 104 session 5: Azure networking
Az 104 session 5: Azure networking
 
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017
 
Type of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleType of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 example
 
Infrastructure Security by Sivamurthy Hiremath
Infrastructure Security by Sivamurthy HiremathInfrastructure Security by Sivamurthy Hiremath
Infrastructure Security by Sivamurthy Hiremath
 
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech Talks
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech TalksAmazon GuardDuty - Let's Attack My Account! - AWS Online Tech Talks
Amazon GuardDuty - Let's Attack My Account! - AWS Online Tech Talks
 
データ活用を加速するAWS分析サービスのご紹介
データ活用を加速するAWS分析サービスのご紹介データ活用を加速するAWS分析サービスのご紹介
データ活用を加速するAWS分析サービスのご紹介
 
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイント
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイントJPC2018[E3]NoOps で進める! Microsoft 365 提案のポイント
JPC2018[E3]NoOps で進める! Microsoft 365 提案のポイント
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEM
 
Cassandraのバックアップと運用を考える
Cassandraのバックアップと運用を考えるCassandraのバックアップと運用を考える
Cassandraのバックアップと運用を考える
 
MITRE ATT&CK Framework
MITRE ATT&CK FrameworkMITRE ATT&CK Framework
MITRE ATT&CK Framework
 
Introducción microsoft azure
Introducción microsoft azureIntroducción microsoft azure
Introducción microsoft azure
 
Splunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-On
 
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術
サイバーエージェントにおけるプライベートコンテナ基盤AKEを支える技術
 
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼
AWS Summit Seoul 2023 | Snowflake: 모든 데이터 워크로드를 위한 하나의 클라우드 데이터 플랫폼
 
AWS CloudFormation Best Practices
AWS CloudFormation Best PracticesAWS CloudFormation Best Practices
AWS CloudFormation Best Practices
 
Azure Networking: Innovative Features and Multi-VNet Topologies
Azure Networking: Innovative Features and Multi-VNet TopologiesAzure Networking: Innovative Features and Multi-VNet Topologies
Azure Networking: Innovative Features and Multi-VNet Topologies
 
20200218 AWS Black Belt Online Seminar Next Generation Redshift
20200218 AWS Black Belt Online Seminar Next Generation Redshift20200218 AWS Black Belt Online Seminar Next Generation Redshift
20200218 AWS Black Belt Online Seminar Next Generation Redshift
 

Destacado

Ca arcserve at cloudian seminar 2013
Ca arcserve at cloudian seminar 2013Ca arcserve at cloudian seminar 2013
Ca arcserve at cloudian seminar 2013CLOUDIAN KK
 
Distributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere MortalsDistributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere MortalsEnsar Basri Kahveci
 
Data Replication in Distributed System
Data Replication in  Distributed SystemData Replication in  Distributed System
Data Replication in Distributed SystemEhsan Hessami
 
Replication in Distributed Database
Replication in Distributed DatabaseReplication in Distributed Database
Replication in Distributed DatabaseAbhilasha Lahigude
 
Dnrec werner arra dba delaware abc 02 04-10 final
Dnrec werner arra dba delaware abc 02 04-10 finalDnrec werner arra dba delaware abc 02 04-10 final
Dnrec werner arra dba delaware abc 02 04-10 finalJim Werner
 
โครงงาน
โครงงานโครงงาน
โครงงานPim Jazz
 
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...vktbox
 
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...vktbox
 
Missouri summary ndaa
Missouri summary ndaaMissouri summary ndaa
Missouri summary ndaaJim Werner
 
Missouri summary ndaa
Missouri summary ndaaMissouri summary ndaa
Missouri summary ndaaJim Werner
 
Opening at cloudian seminar 2012
Opening at cloudian seminar 2012Opening at cloudian seminar 2012
Opening at cloudian seminar 2012CLOUDIAN KK
 
Democamp Munich 2013: Are you still manually coding UIs?
Democamp Munich 2013: Are you still manually coding UIs?Democamp Munich 2013: Are you still manually coding UIs?
Democamp Munich 2013: Are you still manually coding UIs?Maximilian Kögel
 
ZIMSKE OLIMPISKE iGRE SOCI 2014
ZIMSKE OLIMPISKE iGRE SOCI 2014ZIMSKE OLIMPISKE iGRE SOCI 2014
ZIMSKE OLIMPISKE iGRE SOCI 2014PACE Kiprovska
 
Kayla the kiwi
Kayla the kiwiKayla the kiwi
Kayla the kiwigrovedale
 
Taxation of capital gains
Taxation of capital gainsTaxation of capital gains
Taxation of capital gainsNirmal Ghorawat
 
Tpp meningkatkan mutu pendidikan
Tpp  meningkatkan mutu pendidikanTpp  meningkatkan mutu pendidikan
Tpp meningkatkan mutu pendidikanrosid tamami tamami
 

Destacado (20)

Ca arcserve at cloudian seminar 2013
Ca arcserve at cloudian seminar 2013Ca arcserve at cloudian seminar 2013
Ca arcserve at cloudian seminar 2013
 
Distributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere MortalsDistributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere Mortals
 
Data Replication in Distributed System
Data Replication in  Distributed SystemData Replication in  Distributed System
Data Replication in Distributed System
 
Replication in Distributed Database
Replication in Distributed DatabaseReplication in Distributed Database
Replication in Distributed Database
 
Dnrec werner arra dba delaware abc 02 04-10 final
Dnrec werner arra dba delaware abc 02 04-10 finalDnrec werner arra dba delaware abc 02 04-10 final
Dnrec werner arra dba delaware abc 02 04-10 final
 
Iowa
IowaIowa
Iowa
 
Alaska
AlaskaAlaska
Alaska
 
โครงงาน
โครงงานโครงงาน
โครงงาน
 
Jobmatching udine
Jobmatching udineJobmatching udine
Jobmatching udine
 
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...
Vietnam Multi-Activity Tour Travel Hanoi Mai Chau Tam Coc Halong Bay Saigon M...
 
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...
North Vietnam overland tour Travel Hanoi Mai Chau Tam Coc Ba Be Ha Giang Halo...
 
Missouri summary ndaa
Missouri summary ndaaMissouri summary ndaa
Missouri summary ndaa
 
Missouri summary ndaa
Missouri summary ndaaMissouri summary ndaa
Missouri summary ndaa
 
Opening at cloudian seminar 2012
Opening at cloudian seminar 2012Opening at cloudian seminar 2012
Opening at cloudian seminar 2012
 
Democamp Munich 2013: Are you still manually coding UIs?
Democamp Munich 2013: Are you still manually coding UIs?Democamp Munich 2013: Are you still manually coding UIs?
Democamp Munich 2013: Are you still manually coding UIs?
 
ZIMSKE OLIMPISKE iGRE SOCI 2014
ZIMSKE OLIMPISKE iGRE SOCI 2014ZIMSKE OLIMPISKE iGRE SOCI 2014
ZIMSKE OLIMPISKE iGRE SOCI 2014
 
Kayla the kiwi
Kayla the kiwiKayla the kiwi
Kayla the kiwi
 
8.- 9.kl.
8.- 9.kl.8.- 9.kl.
8.- 9.kl.
 
Taxation of capital gains
Taxation of capital gainsTaxation of capital gains
Taxation of capital gains
 
Tpp meningkatkan mutu pendidikan
Tpp  meningkatkan mutu pendidikanTpp  meningkatkan mutu pendidikan
Tpp meningkatkan mutu pendidikan
 

Similar a Cloudian dynamic consistency

Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012CLOUDIAN KK
 
Coherence 12.1.2 Hidden Gems
Coherence 12.1.2 Hidden GemsCoherence 12.1.2 Hidden Gems
Coherence 12.1.2 Hidden Gemsharvraja
 
Choosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform StrategyChoosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform Strategydrmarcustillett
 
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...SQLExpert.pl
 
Sql azure introduction
Sql azure introductionSql azure introduction
Sql azure introductionSuherman .
 
Architecture Best Practices on Windows Azure
Architecture Best Practices on Windows AzureArchitecture Best Practices on Windows Azure
Architecture Best Practices on Windows AzureNuno Godinho
 
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...SL Corporation
 
Cloud Computing : Security and Forensics
Cloud Computing : Security and ForensicsCloud Computing : Security and Forensics
Cloud Computing : Security and ForensicsGovind Maheswaran
 
EMEA OpenStack Day Intro, July 13th 2011 in London
EMEA OpenStack Day Intro, July 13th 2011 in LondonEMEA OpenStack Day Intro, July 13th 2011 in London
EMEA OpenStack Day Intro, July 13th 2011 in LondonMark Collier
 
Scalar DL Technical Overview
Scalar DL Technical OverviewScalar DL Technical Overview
Scalar DL Technical OverviewScalar, Inc.
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSFAdrian Cockcroft
 
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry intro
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry introEMEA OpenStack Day, July 13th 2011 in London - Jim Curry intro
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry introOpen Stack
 
Safe Peak Technical Ppt W Product Publish
Safe Peak Technical Ppt W Product   PublishSafe Peak Technical Ppt W Product   Publish
Safe Peak Technical Ppt W Product Publishsqlserver.co.il
 
OSS Presentation by Stefano Maffulli
OSS Presentation by Stefano MaffulliOSS Presentation by Stefano Maffulli
OSS Presentation by Stefano MaffulliOpenStorageSummit
 
Sql server 2012 - always on deep dive - bob duffy
Sql server 2012 - always on deep dive - bob duffySql server 2012 - always on deep dive - bob duffy
Sql server 2012 - always on deep dive - bob duffyAnuradha
 
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018Amazon Web Services Korea
 
Architecture Openstack for the Enterprise
Architecture Openstack for the EnterpriseArchitecture Openstack for the Enterprise
Architecture Openstack for the EnterpriseKeith Tobin
 
CCCC Neustar Lenny Rachitsky
CCCC Neustar Lenny RachitskyCCCC Neustar Lenny Rachitsky
CCCC Neustar Lenny RachitskyCloud Congress
 
The sky's the limit
The sky's the limitThe sky's the limit
The sky's the limitIvan Zoratti
 

Similar a Cloudian dynamic consistency (20)

Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
 
Coherence 12.1.2 Hidden Gems
Coherence 12.1.2 Hidden GemsCoherence 12.1.2 Hidden Gems
Coherence 12.1.2 Hidden Gems
 
Choosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform StrategyChoosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform Strategy
 
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...
 
Sql azure introduction
Sql azure introductionSql azure introduction
Sql azure introduction
 
Architecture Best Practices on Windows Azure
Architecture Best Practices on Windows AzureArchitecture Best Practices on Windows Azure
Architecture Best Practices on Windows Azure
 
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, D...
 
Cloud Computing : Security and Forensics
Cloud Computing : Security and ForensicsCloud Computing : Security and Forensics
Cloud Computing : Security and Forensics
 
Backup & Disaster Recovery on AWS - An overview of our Approach
Backup & Disaster Recovery on AWS - An overview of our ApproachBackup & Disaster Recovery on AWS - An overview of our Approach
Backup & Disaster Recovery on AWS - An overview of our Approach
 
EMEA OpenStack Day Intro, July 13th 2011 in London
EMEA OpenStack Day Intro, July 13th 2011 in LondonEMEA OpenStack Day Intro, July 13th 2011 in London
EMEA OpenStack Day Intro, July 13th 2011 in London
 
Scalar DL Technical Overview
Scalar DL Technical OverviewScalar DL Technical Overview
Scalar DL Technical Overview
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSF
 
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry intro
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry introEMEA OpenStack Day, July 13th 2011 in London - Jim Curry intro
EMEA OpenStack Day, July 13th 2011 in London - Jim Curry intro
 
Safe Peak Technical Ppt W Product Publish
Safe Peak Technical Ppt W Product   PublishSafe Peak Technical Ppt W Product   Publish
Safe Peak Technical Ppt W Product Publish
 
OSS Presentation by Stefano Maffulli
OSS Presentation by Stefano MaffulliOSS Presentation by Stefano Maffulli
OSS Presentation by Stefano Maffulli
 
Sql server 2012 - always on deep dive - bob duffy
Sql server 2012 - always on deep dive - bob duffySql server 2012 - always on deep dive - bob duffy
Sql server 2012 - always on deep dive - bob duffy
 
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
 
Architecture Openstack for the Enterprise
Architecture Openstack for the EnterpriseArchitecture Openstack for the Enterprise
Architecture Openstack for the Enterprise
 
CCCC Neustar Lenny Rachitsky
CCCC Neustar Lenny RachitskyCCCC Neustar Lenny Rachitsky
CCCC Neustar Lenny Rachitsky
 
The sky's the limit
The sky's the limitThe sky's the limit
The sky's the limit
 

Más de CLOUDIAN KK

CLOUDIAN HYPERSTORE - 風林火山ストレージ
CLOUDIAN HYPERSTORE - 風林火山ストレージCLOUDIAN HYPERSTORE - 風林火山ストレージ
CLOUDIAN HYPERSTORE - 風林火山ストレージCLOUDIAN KK
 
クラウディアンのご紹介
クラウディアンのご紹介クラウディアンのご紹介
クラウディアンのご紹介CLOUDIAN KK
 
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革CLOUDIAN KK
 
CLOUDIAN Presentation at VERITAS VISION in Tokyo
CLOUDIAN Presentation at VERITAS VISION in TokyoCLOUDIAN Presentation at VERITAS VISION in Tokyo
CLOUDIAN Presentation at VERITAS VISION in TokyoCLOUDIAN KK
 
S3 API接続検証プログラムのご紹介
S3 API接続検証プログラムのご紹介S3 API接続検証プログラムのご紹介
S3 API接続検証プログラムのご紹介CLOUDIAN KK
 
Auto tiering and Versioning of CLOUDIAN HyperStore
Auto tiering and Versioning of CLOUDIAN HyperStoreAuto tiering and Versioning of CLOUDIAN HyperStore
Auto tiering and Versioning of CLOUDIAN HyperStoreCLOUDIAN KK
 
AWS SDK for Python and CLOUDIAN HyperStore
AWS SDK for Python and CLOUDIAN HyperStoreAWS SDK for Python and CLOUDIAN HyperStore
AWS SDK for Python and CLOUDIAN HyperStoreCLOUDIAN KK
 
AWS CLI and CLOUDIAN HyperStore
AWS CLI and CLOUDIAN HyperStoreAWS CLI and CLOUDIAN HyperStore
AWS CLI and CLOUDIAN HyperStoreCLOUDIAN KK
 
ZiDOMA data and CLOUDIAN HyperStore
ZiDOMA data and CLOUDIAN HyperStoreZiDOMA data and CLOUDIAN HyperStore
ZiDOMA data and CLOUDIAN HyperStoreCLOUDIAN KK
 
FOBAS CSC and CLOUDIAN HyperStore
FOBAS CSC and CLOUDIAN HyperStoreFOBAS CSC and CLOUDIAN HyperStore
FOBAS CSC and CLOUDIAN HyperStoreCLOUDIAN KK
 
ARCserve backup and CLOUDIAN HyperStore
ARCserve backup and CLOUDIAN HyperStoreARCserve backup and CLOUDIAN HyperStore
ARCserve backup and CLOUDIAN HyperStoreCLOUDIAN KK
 
Cloudian presentation at idc japan sv2016
Cloudian presentation at idc japan sv2016Cloudian presentation at idc japan sv2016
Cloudian presentation at idc japan sv2016CLOUDIAN KK
 
ITコアを刷新するハイブリッドクラウド型ITシステム
ITコアを刷新するハイブリッドクラウド型ITシステムITコアを刷新するハイブリッドクラウド型ITシステム
ITコアを刷新するハイブリッドクラウド型ITシステムCLOUDIAN KK
 
【FOBAS】Data is money. ストレージ分散投資のススメ
【FOBAS】Data is money. ストレージ分散投資のススメ【FOBAS】Data is money. ストレージ分散投資のススメ
【FOBAS】Data is money. ストレージ分散投資のススメCLOUDIAN KK
 
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較CLOUDIAN KK
 
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化CLOUDIAN KK
 
【CLOUDIAN】コード化されたインフラの実装
【CLOUDIAN】コード化されたインフラの実装【CLOUDIAN】コード化されたインフラの実装
【CLOUDIAN】コード化されたインフラの実装CLOUDIAN KK
 
【CLOUDIAN】自動階層化による現有ストレージ活用術
【CLOUDIAN】自動階層化による現有ストレージ活用術【CLOUDIAN】自動階層化による現有ストレージ活用術
【CLOUDIAN】自動階層化による現有ストレージ活用術CLOUDIAN KK
 
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現CLOUDIAN KK
 
【Cloudian】FIT2015における会社製品紹介
【Cloudian】FIT2015における会社製品紹介【Cloudian】FIT2015における会社製品紹介
【Cloudian】FIT2015における会社製品紹介CLOUDIAN KK
 

Más de CLOUDIAN KK (20)

CLOUDIAN HYPERSTORE - 風林火山ストレージ
CLOUDIAN HYPERSTORE - 風林火山ストレージCLOUDIAN HYPERSTORE - 風林火山ストレージ
CLOUDIAN HYPERSTORE - 風林火山ストレージ
 
クラウディアンのご紹介
クラウディアンのご紹介クラウディアンのご紹介
クラウディアンのご紹介
 
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革
IoT/ビッグデータ/AI連携により次世代ストレージが促進するビジネス変革
 
CLOUDIAN Presentation at VERITAS VISION in Tokyo
CLOUDIAN Presentation at VERITAS VISION in TokyoCLOUDIAN Presentation at VERITAS VISION in Tokyo
CLOUDIAN Presentation at VERITAS VISION in Tokyo
 
S3 API接続検証プログラムのご紹介
S3 API接続検証プログラムのご紹介S3 API接続検証プログラムのご紹介
S3 API接続検証プログラムのご紹介
 
Auto tiering and Versioning of CLOUDIAN HyperStore
Auto tiering and Versioning of CLOUDIAN HyperStoreAuto tiering and Versioning of CLOUDIAN HyperStore
Auto tiering and Versioning of CLOUDIAN HyperStore
 
AWS SDK for Python and CLOUDIAN HyperStore
AWS SDK for Python and CLOUDIAN HyperStoreAWS SDK for Python and CLOUDIAN HyperStore
AWS SDK for Python and CLOUDIAN HyperStore
 
AWS CLI and CLOUDIAN HyperStore
AWS CLI and CLOUDIAN HyperStoreAWS CLI and CLOUDIAN HyperStore
AWS CLI and CLOUDIAN HyperStore
 
ZiDOMA data and CLOUDIAN HyperStore
ZiDOMA data and CLOUDIAN HyperStoreZiDOMA data and CLOUDIAN HyperStore
ZiDOMA data and CLOUDIAN HyperStore
 
FOBAS CSC and CLOUDIAN HyperStore
FOBAS CSC and CLOUDIAN HyperStoreFOBAS CSC and CLOUDIAN HyperStore
FOBAS CSC and CLOUDIAN HyperStore
 
ARCserve backup and CLOUDIAN HyperStore
ARCserve backup and CLOUDIAN HyperStoreARCserve backup and CLOUDIAN HyperStore
ARCserve backup and CLOUDIAN HyperStore
 
Cloudian presentation at idc japan sv2016
Cloudian presentation at idc japan sv2016Cloudian presentation at idc japan sv2016
Cloudian presentation at idc japan sv2016
 
ITコアを刷新するハイブリッドクラウド型ITシステム
ITコアを刷新するハイブリッドクラウド型ITシステムITコアを刷新するハイブリッドクラウド型ITシステム
ITコアを刷新するハイブリッドクラウド型ITシステム
 
【FOBAS】Data is money. ストレージ分散投資のススメ
【FOBAS】Data is money. ストレージ分散投資のススメ【FOBAS】Data is money. ストレージ分散投資のススメ
【FOBAS】Data is money. ストレージ分散投資のススメ
 
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較
【ARI】ストレージのコスト・利便性・非機能要求項目を徹底比較
 
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化
【SIS】オブジェクトストレージを活用した増え続ける長期保管データの運用の効率化
 
【CLOUDIAN】コード化されたインフラの実装
【CLOUDIAN】コード化されたインフラの実装【CLOUDIAN】コード化されたインフラの実装
【CLOUDIAN】コード化されたインフラの実装
 
【CLOUDIAN】自動階層化による現有ストレージ活用術
【CLOUDIAN】自動階層化による現有ストレージ活用術【CLOUDIAN】自動階層化による現有ストレージ活用術
【CLOUDIAN】自動階層化による現有ストレージ活用術
 
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現
【CLOUDIAN】秒間隔RPO(目標復旧時点)の実現
 
【Cloudian】FIT2015における会社製品紹介
【Cloudian】FIT2015における会社製品紹介【Cloudian】FIT2015における会社製品紹介
【Cloudian】FIT2015における会社製品紹介
 

Cloudian dynamic consistency

  • 1. Dynamic Consistency April 2013 Cloudian KK & Inc. (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved. 1
  • 2. Purpose of this document Cloudian developed “Dynamic Consistency” and released it in Cloudian version 2.4. “Dynamic Consistency” is a unique technology that dynamically changes data consistency in a high-availability configuration Provides the following benefits: - Continue services in case of failures without IT intervention - Maintain highest possible data consistency This document covers the following topics: What is data consistency in a distributed system? What is the problem with static data consistency? How does “Dynamic Consistency” solve the problem? 2 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 3. What is “Cloudian”? Cloudian is packaged software product that provides; Distributed object storage system Amazon S3 compatible API Unlimited scalability to hundreds of petabytes Data replications in multi-data centers Tunable data consistency Cloudian is being used by large organizations to offer cloud storage services/systems 3 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 4. How Replication works in the distributed system? One object is replicated to Even in the case of a multiple servers server failure, client can get the object Client Client Put “object” Get “object” Server Server Server Server Server Server 1 2 3 1 2 3 Replication 4 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 5. Cloudian “Data Consistency” levels are tunable Write Consistency Read Consistency 5 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 6. How “Data Consistency” works in the distributed system? After “all” objects are After “some” objects are replicated, Client receives replicated, Client receives acknowledgement acknowledgement Client All example Client QUORUM(R+W>N) example (N) Replications=3 (N) Replications=3 (W) Write=3 (W) Write=2 (R) Read=any (R) Read=2 “ack” “ack” Server Server Server Server Server Server 1 2 3 1 2 3 Client Client Data is always consistent Data is consistent by 6 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved. QUORUM
  • 7. Problem: “All” is disruptive in case of failure & slow for replication over slow links “All” or “QUORUM” cannot be completed in the case of failure “All” or “QUORUM” is slower for clients/applications Assumptions: • 3 Replications • 2 in DC1 Client • 1 in DC2 No “ack” Server Server Server Server Server Server 1 2 3 4 5 6 Data Center 1 Data Center 2 7 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 8. Problem: “Local_QUORUM ” does not guarantee “Data Consistent” in the whole system “Local _Quorum” cannot provide “data consistency” in the whole system Assumptions: Client Client Client may get “object” • 3 Replications before updated • 2 in DC1 • 1 in DC2 “ack” Server Server Server Server Server Server 1 2 3 4 5 6 Data Center 1 Data Center 2 Data is consistent only when object is read from DC1 8 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 9. “Cloudian” provides Dynamic & Flexible Consistency Synchronous Asynchronous Replication Replication Traditional Approach (static & + No Data Loss + Not dependent on inflexible) - Failures if DR site availability of DR site not reachable - Data Loss in case of Disaster Synchronous Each Local Quorum Asynchronous Cloudian Replication Quorum Quorum Replication Multi-DC (static & flexible) Cloudian 2.4 Synchronous Each Local Quorum Asynchronous Multi-DC Replication #1 Quorum Quorum Replication #3 (dynamic & #2 flexible) 9 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 10. Solution: “Dynamic Consistency” changes consistency level dynamically (1/2) 1. System operates at “All” to keep “Data Consistency” Assumptions: Client Client • 3 Replications • 2 in DC1 • 1 in DC2 Client always get “ack” the same “object” Server Server Server Server Server Server 1 2 3 4 5 6 Data Center 1 Data Center 2 Data is consistent in the whole system 10 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 11. Solution: “Dynamic Consistency” changes consistency level dynamically (2/2) 2. System automatically falls back to “Local_QUORUM” in case of data center failure Assumptions: • 3 Replications • 2 in DC1 Client Client • 1 in DC2 Client gets “object” from “ack” surviving data center Server Server Server Server Server Server 1 2 3 4 5 6 Data Center 1 Data Center 2 11 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.
  • 12. For more information: Contact by Email: info@cloudian.com Homepage: www.cloudian.com www.cloudian.jp Cloudian Community Edition: http://www.cloudian.com/get-started.html 30⽇間無料トライル: http://www.cloudian.jp/cloud-storage-products/cloudian_eval_agr.php 12 (c) Copyright , Cloudian KK&Inc. 2013, All rights reserved.