Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Oracle 12.2 sharded database management

7.050 visualizaciones

Publicado el

Oracle 12.2 (Beta) sharded database management

Publicado en: Tecnología

Oracle 12.2 sharded database management

  1. 1. NEED SHARDING? HERE IT IS! Oracle 12.2 Sharded Database Management
  2. 2. ABOUT ME ➤ Kamus@Enmotech 张乐奕 ➤ Travel and Starbucks ➤ Travel: Japan ➤ Starbucks: Venti Caramel Macchiato ➤ Games: Blizzard fans, Hearthstone ➤ Channel[K]: http://www.dbform.com
  3. 3. WHATISSHARDING Every shard is a part of logical database
  4. 4. WHAT IS SHARDING EXACTLY //Query single row select User_Name from T1 where User_ID=1; //Query all create view T as select * from T1 union all select * from T2; select count(*) from T; Database “DB” Database “DB” Split Table Table “T” User_ID User_Name 1 Kim 2 Tim 3 Jim 4 Sim Table “T1” User_ID User_Name 1 Kim 3 Jim Table “T2” User_ID User_Name 2 Tim 4 Sim
  5. 5. WHAT IS SHARDING EXACTLY //Query single row select User_Name from T where User_ID=1; //Query all select count(*) from T; Database “DB” Database “DB” Partition Table Table “T” User_ID User_Name 1 Kim 2 Tim 3 Jim 4 Sim Partition “P1” User_ID User_Name 1 Kim 3 Jim Partition“P2” User_ID User_Name 2 Tim 4 Sim Table “T”
  6. 6. WHAT IS SHARDING EXACTLY //Query single row select User_Name from T where User_ID=1; //Query all select count(*) from T; Database “DB” Database “DB1” Shard Table Table “T” User_ID User_Name 1 Kim 2 Tim 3 Jim 4 Sim Shard “S1” User_ID User_Name 1 Kim 3 Jim Shard “S2” User_ID User_Name 2 Tim 4 Sim Database “DB2” Table “T”
  7. 7. WHY NEED SHARDING? BENEFITS OF SHARDING ➤ Extreme Scalability ➤ Scale Out vs. Scale Up ➤ Fault Isolation ➤ No shared SAN ➤ Global Data Distribution ➤ Store particular data close to its consumers ➤ Pilot & Rolling Upgrades ➤ First test the changes on a small subset of data. ➤ Cost Down ➤ Because the size of a shard can be made arbitrarily small, deploying an SDB in a cloud consisting of low-end commodity servers with local storage becomes easy. 

  8. 8. DATABASE SPLITTING HISTORY IN WEB COMPANIES ➤ All business on Single database ➤ All business on Master+Slave databases, write/read separated. ➤ Business separated, each business on separated Master+Slave databases. ➤ Area based/Hash based, table horizontal split, each part of business on separated Master+Slave databases. Database “DB” Write Read Master Database “DB1” Write Slave Database “DB2” Read Replication Master Database “DB1-1” Write Slave Database “DB1-2” Read Replication Master Database “DBN-1” Write Slave Database “DBN-2” Read Replication Master Database “DB1-1” Write Slave Database “DB1-2” Read Replication Master Database “DB1-1” Write Slave Database “DB1-2” Read Replication Master Database “DB1-1” Write Slave Database “DB1-2” Read Replication Master Database “DBN-1” Write Slave Database “DBN-2” Read Replication Master Database “DBN-1” Write Slave Database “DBN-2” Read Replication Master Database “DBN-1” Write Slave Database “DBN-2” Read Replication
  9. 9. DISADVANTAGES OF SHARDING ➤ Is it transparent to application developer? ➤ Which database should access? ➤ How to sum query all databases? ➤ How to join query all databases? Database access layer/Data access route layer ➤ Proxy-Free:MySQL Fabric, TDDL ➤ Proxy:MySQL Proxy, Atlas(Qihoo),Cobar } ➤ Single point of failure ➤ Failover more complex ➤ Backups more complex ➤ Modification more complex
  10. 10. HOW MANY SHARDING SOLUTIONS? ➤ SQL ➤ MySQL with MySQL Fabric ➤ Postgres-XC and Greenplum ➤ Teradata ➤ No-SQL ➤ Apache HBase ➤ MongoDB ➤ New-SQL ➤ Google Spanner And NOW is coming!
  11. 11. WHAT IS ORACLE SHARDING Every shard is a part of logical database
  12. 12. GLOBAL DATA SERVICE - GDS ➤ What is GDS ➤ Instance->Service->Global Service ➤ GDS Capabilities ➤ Workload routing(region-based or lag-based) ➤ Load balancing(Connect-time or Run-time) ➤ Global service failover/role-based failover ➤ Released in Oracle database 12.1 ➤ GSM(Global Service Manager) ➤ GDS Catalog ➤ Must reside in Oracle database 12c ➤ Recommended to co-hosted with RMAN catalog or OEM repository
  13. 13. GDS DEPLOYMENT OVERVIEW ➤ Install GSM software on GSM servers ➤ Minimum 1 GSM per region ➤ Recommended 3 GSMs/region ➤ Setup GDS administrator accounts & privileges ➤ Configure GDS ➤ Create GDS catalog ➤ Add GSMs,Regions, Pools, Databases, Global Services ➤ Setup client connectivity ➤ Clients connect to GSM instead of the database listener
  14. 14. GDS USE CASES - ROLE BASED GLOBAL SERVICES FOR ACTIVE DATA GUARD ➤ Order Service runs on Primary ➤ Reporting Service runs on Standby ➤ Upon Data Guard role change, GDS fails over services based on Role GDSCTL> add service -service order_srvc -gdspool sales - preferred_all -role PRIMAY; GDSCTL> add service -service reporting_srvc -gdspool sales -preferred_all -role PHYSICAL_STANBY -failover_primay;
  15. 15. GDS USE CASES - REPLICATION LAG TOLERANCE IN ACTIVE DATA GUARD ➤ Specify replication lag limit for a service ➤ GDS ensures that service runs on Active Data Guard standby(s) with lag less than this limit ➤ Improved data quality GDSCTL> add service -service reporting_srvc -gdspool sales –preferred_all –role PHYSICAL_STANDBY –lag 180;
  16. 16. ORACLE SHARDING USE CASE Oracle Sharding is implemented based on the Oracle Database partitioning feature. Oracle Sharding is “Distribute Partitioning”.
  17. 17. ORACLE SHARDING KNOWHOW ➤ Table family ➤ Parent-child relationship tables ➤ In Oracle Database 12c Release 2, only a single table family is supported in an SDB Sharding Key
  18. 18. ORACLE SHARDING KNOWHOW ➤ Consistent Hash ➤ Traditional hash: bucket number is calculated as HF(key) % N ➤ Oracle linear hashing: buckets number is power of 2, if not, data is not even. ➤ Consistent hashing: partitioning by the range of hash values ➤ When add/remove shards, only move chunks, does not change contents of chunks ➤ When chunk is split, hash values divide into 2 ranges, other chunks no infect
  19. 19. ORACLE SHARDING KNOWHOW ➤ Tablespace set ➤ Before creating a sharded table partitioned by consistent hash, a set of tablespaces (one tablespace per chunk) has to be created to store the table partitions. ➤ Chunk ➤ A chunk contains a single partition from each table of a table family ➤ Default, 120 chunks per shard, or specified when CREATE SHARDCATALOG by using GDSCTL ➤ The unit of data migration between shards is the chunk. Customers_P1(1-1000000) Orders_P1 Lineitems_P1Chunk #1
  20. 20. ORACLE SHARDING KNOWHOW ➤ Duplicated table ➤ Synchronizes the contents of duplicated tables using Materialized View Replication ➤ A duplicated table on each shard is represented by a read-only materialized view. ➤ Automatically refreshed with a configurable frequency ➤ CREATE DUPLICATED TABLE statement automatically creates the master table, materialized views, and other objects required for materialized view replication.
  21. 21. SHARDING AND REPLICATION ➤ Managing by using GDSCTL ➤ Automatically deploys ➤ Data Guard ➤ Goldengate //System-Managed Sharding with Data Guard Replication CREATE SHARDCATALOG –database host00:1521:shardcat –region dc1, dc2 ADD SHARDGROUP -shardgroup shardgroup1 -region dc1 -deploy_as primary ADD SHARDGROUP -shardgroup shardgroup2 -region dc1 -deploy_as standby ADD SHARDGROUP -shardgroup shardgroup3 -region dc2 -deploy_as standby CREATE SHARD … DEPLOY Database “DB1” Shard “S1” User_ID User_Name 1 Kim 3 Jim Shard “S2” User_ID User_Name 2 Tim 4 Sim Database “DB2” Table “T” Shardgroup 1 Database “DB1” Shard “S1” User_ID User_Name 1 Kim 3 Jim Shard “S2” User_ID User_Name 2 Tim 4 Sim Database “DB2” Table “T” Shardgroup 2 Database “DB1” Shard “S1” User_ID User_Name 1 Kim 3 Jim Shard “S2” User_ID User_Name 2 Tim 4 Sim Database “DB2” Table “T” Shardgroup 3 DC 1 DC 2 //System-Managed Sharding with Goldengate Replication CREATE SHARDCATALOG -repl OGG –database host00:1521:shardcat –region dc1, dc2 ADD SHARDGROUP -shardgroup shardgroup1 -region dc1 ADD SHARDGROUP -shardgroup shardgroup2 -region dc2 CREATE SHARD … DEPLOY
  22. 22. SHARDING REQUEST ROUTING ➤ Session-Based Request Routing ➤ Connect string must contain: (SHARD_KEY=...) ➤ JDBC: connection.setShardKey(<shard_key>,<shard_group_key>); ➤ Support for OCI/OCCI (C++)/ODP.NET ➤ Support for PHP, Python, Perl, and Node.js ➤ In addition to Session-Based routing, Oracle also provide support for: ➤ Statement-Level Request Routing ➤ Cross-Shard Queries
  23. 23. “Unlike NoSQL data stores, Oracle Sharding provides the benefits of sharding without sacrificing the capabilities of an enterprise RDBMS, such as relational schema, SQL and other programmatic interfaces, complex data types, online schema changes, multi- core scalability, advanced security, compression, high- availability, ACID properties, consistent reads, and many more.. -Oracle® Database Administrator's Guide 12c Release 2 (12.2)
  24. 24. THINK TWICE BEFORE JUMPING INTO SHARDING ➤ MySQL: Not exceed 20GB per table is optimised. (Concurrent processing capacity) ➤ How about we have a 2TB table? 100 shards! ➤ What does 100 shards mean? ➤ 100 x86 servers ➤ 1 Primary, 1 Slave -> 200 servers ➤ 1 Primary, 2 Slaves -> 300 servers ➤ How many routers do you need? How many cables? How many spaces? ➤ Are you ready for maintaining so many servers? ➤ 99% server stableness means: Server down/EVERY day.
  25. 25. LAST CONCERN ➤ License cost ➤ Oracle database license model has to be changed 32 CPU,$1,152,000/Y 32 CPU,$1,958,400/Y

×