Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Polyglot
Persistence
Majid Fatemian
R E D V E N T U R E S
@majidfn
Relational Non-Relational Key-Value Big Data
Application
Relational
Application
Relational Relational Non-Relational
Relational
BIG
DATA
Capture
Search
Storage
Q u e r y
Privacy
Visualize
Analysis
Non-Relational
1 2 3
Non-Relational Cluster Schemaless
2 x
1 x
1 x
2011
2012
2013
2014
2011
2012
2013
2014
Application
ACID
Normalized
{Consistency
Relational
Player ID Map ID Difficulty Score
1000 223 E 5
2000 673 H 4.5
1000 451 M 4
2000 980 H 3
Activity
Player ID Name Location
100...
Replication
Fault Tolerance
Availability
{Relational
OPs / DBAs
Tools
Devs{DevOps
Relational
2011
2012
2013
2014
Scalability
Relational
Distribution
Relational
Data C
Data B
Data A
Data Access Layer
Schema
Relational
{

player_id: 1234,

* level: 40,

* duration: 75,
* activity: activity,

game_id: 4001,

platform: WiiU...
Total
Detailed
stats
stats
Players Profiles
Individual Activities
Aggregated
stats
Categorized
ACID
Transaction
Partitioning
Replication
Tools
OPs
Data Volume
Fixed Schema
Scale Up
Manual Sharding
+ -
NoSQL
...
mongoDB
1 2 3
OPs Devs Community
Schema-less
Non-Relational
player_id: 1234,

* level: 40,

* duration: 75,
* activity: activity,
game_id: 4001,

platform:...
Scalability
Non-Relational
Scalability
Non-Relational
+
Sharding
Non-Relational
Data A
{key}
Sharding
Non-Relational
Data A
A
Data B
B
Data C
C
{key}
Map/Reduce
Non-Relational
Query
K, V
K,V1 K,V2 K,V3
Map/Reduce
Non-Relational
Map
K,V1 K,V2 K,V3
Map/Reduce
Non-Relational
Result
ReduceMap
K,V1 K,V2 K,V3
Denormalized
Non-Relationalread /
}write
Disk Space
Player ID Map ID Difficulty Score
1000 223 E 5
2000 673 H 4.5
1000 451 M 4
2000 980 H 3
Activity
Player ID Name Location
100...
{Name: John, game:1, activity: Mission01, Score: 1245}
{Name: John, game:3, activity: Mission04, score: 5431}
{Name: John,...
Data Model
Non-RelationalQuery
Aggregation
}
Eventual-
Consistency
Non-Relational
X2
W
X1 X1
X2
Eventual-
Consistency
Non-Relational
X2
W
X1 X1
R
X1
X2
Eventual-
Consistency
Non-Relational
X2
W
X2
X2
R
X2
X2
Balancer
Non-Relational
Balancer
Non-Relational
+
Balancer
Non-Relational
Non-Relational
Resilience
A B C
Non-Relational
Resilience
A C
Locking
Non-RelationalDB Level (2.2)
}Document Level (3.x)
Backup
Non-Relational
Point-in-time
Denormalized data
Disk Space
Expertise
Complex Querying

Eventual Consistency
Resource Usage
DB-Level Locking
-
Schema-Les...
Random Profile

Search

(1M)
Full History
Retrieval

(6M)
Insert

(6M)
Aggregation

Map/Reduce

(6M)
0
0.003
0.005
0.008
0....
Persistence
Polyglot
Players Profile
Leaderboards
Activity Details
Relational Data
Schema-less
Sharding
Total Sum
Activity 1
Activity n
Activity m
……
Shard1
Shard2
Config
Data Access Layer
Application
The
Challenge
Inconsistency1
Data Access Layer
W
Pending
Commit
2-Phase
Data Access Layer
Pending
W
Commit
2-Phase
Data Access Layer
Done
Commit
2-Phase
2 Latency
vs. Consistency
Operations3 Exponential complexity
Availability4 Plan for failure
Data Access Layer
Data Access Layer
Data Access Layer
Data Integrity5 Backward compatibility
Unit
{TEST Integration
Functional
Data Migration6
Profile
History }
{
profile:
{
GUID:…,
platform:
{
name:’xbox’,
xuid:…
}
},
activity:…,
duration:…,
datetime:…
}
3rd Party
1 One-Shot Migration
4 hours !!
Live !!
2 Profile versioning
V
+1
1
V
+1
1
V
+1
2
Final
Results
Scalability
2011 2012 2013 2014
9x response time
3 years uptime
Go Hybrid!
Thoughtfully
>
Thank
YOU!
@majidfn
References
Polyglot Persistence
NoSQL Distilled
Polyglot Persistence - Confoo 2016
Polyglot Persistence - Confoo 2016
Polyglot Persistence - Confoo 2016
Polyglot Persistence - Confoo 2016
Polyglot Persistence - Confoo 2016
Polyglot Persistence - Confoo 2016
Próxima SlideShare
Cargando en…5
×

Polyglot Persistence - Confoo 2016

10.002 visualizaciones

Publicado el

NoSQL databases are every where and they are here to stay. For a successful scalable web application it is vital to understand capabilities of NoSQL databases.

Also relational databases are not dead and they will not be. Understanding abilities of each, would help making better architectural decisions.

Hybrid database solutions could use bests of the both worlds. We would go in to the important decision making factors for each.

Publicado en: Internet
  • Follow the link, new dating source: ❤❤❤ http://bit.ly/2u6xbL5 ❤❤❤
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Dating direct: ❶❶❶ http://bit.ly/2u6xbL5 ❶❶❶
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Sé el primero en recomendar esto

Polyglot Persistence - Confoo 2016

  1. 1. Polyglot Persistence
  2. 2. Majid Fatemian R E D V E N T U R E S @majidfn
  3. 3. Relational Non-Relational Key-Value Big Data
  4. 4. Application Relational
  5. 5. Application Relational Relational Non-Relational
  6. 6. Relational
  7. 7. BIG DATA
  8. 8. Capture Search Storage Q u e r y Privacy Visualize Analysis
  9. 9. Non-Relational
  10. 10. 1 2 3 Non-Relational Cluster Schemaless
  11. 11. 2 x 1 x 1 x
  12. 12. 2011 2012 2013 2014
  13. 13. 2011 2012 2013 2014
  14. 14. Application
  15. 15. ACID Normalized {Consistency Relational
  16. 16. Player ID Map ID Difficulty Score 1000 223 E 5 2000 673 H 4.5 1000 451 M 4 2000 980 H 3 Activity Player ID Name Location 1000 John US 2000 Catherine CA 3000 Paul CA 4000 Kylee US Profile
  17. 17. Replication Fault Tolerance Availability {Relational
  18. 18. OPs / DBAs Tools Devs{DevOps Relational
  19. 19. 2011 2012 2013 2014
  20. 20. Scalability Relational
  21. 21. Distribution Relational Data C Data B Data A Data Access Layer
  22. 22. Schema Relational {
 player_id: 1234,
 * level: 40,
 * duration: 75, * activity: activity,
 game_id: 4001,
 platform: WiiU, difficulty_level: medium,
 repetition: 178,
 score: 450,
 endurance:3
 } {
 player_id: 1234,
 level: 12,
 duration: 120, activity: activity
 }
  23. 23. Total Detailed stats stats Players Profiles Individual Activities Aggregated stats Categorized
  24. 24. ACID Transaction Partitioning Replication Tools OPs Data Volume Fixed Schema Scale Up Manual Sharding + -
  25. 25. NoSQL ...
  26. 26. mongoDB 1 2 3 OPs Devs Community
  27. 27. Schema-less Non-Relational player_id: 1234,
 * level: 40,
 * duration: 75, * activity: activity, game_id: 4001,
 platform: WiiU, difficulty_level: 2,
 repetition: 178,
 score: 450,
 endurance:3 player_id: 1234,
 level: 75,
 duration: 120, activity: activity

  28. 28. Scalability Non-Relational
  29. 29. Scalability Non-Relational +
  30. 30. Sharding Non-Relational Data A {key}
  31. 31. Sharding Non-Relational Data A A Data B B Data C C {key}
  32. 32. Map/Reduce Non-Relational Query K, V K,V1 K,V2 K,V3
  33. 33. Map/Reduce Non-Relational Map K,V1 K,V2 K,V3
  34. 34. Map/Reduce Non-Relational Result ReduceMap K,V1 K,V2 K,V3
  35. 35. Denormalized Non-Relationalread / }write Disk Space
  36. 36. Player ID Map ID Difficulty Score 1000 223 E 5 2000 673 H 4.5 1000 451 M 4 2000 980 H 3 Activity Player ID Name Location 1000 John US 2000 Catherine CA 3000 Paul CA 4000 Kylee US Profile
  37. 37. {Name: John, game:1, activity: Mission01, Score: 1245} {Name: John, game:3, activity: Mission04, score: 5431} {Name: John, game:1, activity: Mission01, Score: 1245}
  38. 38. Data Model Non-RelationalQuery Aggregation }
  39. 39. Eventual- Consistency Non-Relational X2 W X1 X1 X2
  40. 40. Eventual- Consistency Non-Relational X2 W X1 X1 R X1 X2
  41. 41. Eventual- Consistency Non-Relational X2 W X2 X2 R X2 X2
  42. 42. Balancer Non-Relational
  43. 43. Balancer Non-Relational +
  44. 44. Balancer Non-Relational
  45. 45. Non-Relational Resilience A B C
  46. 46. Non-Relational Resilience A C
  47. 47. Locking Non-RelationalDB Level (2.2) }Document Level (3.x)
  48. 48. Backup Non-Relational Point-in-time
  49. 49. Denormalized data Disk Space Expertise Complex Querying
 Eventual Consistency Resource Usage DB-Level Locking - Schema-Less Aggregated Data Large Scale Data Sharding Map/Reduce Memory Storage Journaling +
  50. 50. Random Profile
 Search
 (1M) Full History Retrieval
 (6M) Insert
 (6M) Aggregation
 Map/Reduce
 (6M) 0 0.003 0.005 0.008 0.01 MySQL MongoDB
  51. 51. Persistence Polyglot
  52. 52. Players Profile Leaderboards Activity Details Relational Data Schema-less Sharding
  53. 53. Total Sum Activity 1 Activity n Activity m ……
  54. 54. Shard1 Shard2 Config
  55. 55. Data Access Layer Application
  56. 56. The Challenge
  57. 57. Inconsistency1
  58. 58. Data Access Layer W Pending Commit 2-Phase
  59. 59. Data Access Layer Pending W Commit 2-Phase
  60. 60. Data Access Layer Done Commit 2-Phase
  61. 61. 2 Latency vs. Consistency
  62. 62. Operations3 Exponential complexity
  63. 63. Availability4 Plan for failure
  64. 64. Data Access Layer
  65. 65. Data Access Layer
  66. 66. Data Access Layer
  67. 67. Data Integrity5 Backward compatibility
  68. 68. Unit {TEST Integration Functional
  69. 69. Data Migration6
  70. 70. Profile History } { profile: { GUID:…, platform: { name:’xbox’, xuid:… } }, activity:…, duration:…, datetime:… } 3rd Party
  71. 71. 1 One-Shot Migration 4 hours !! Live !! 2 Profile versioning
  72. 72. V +1 1
  73. 73. V +1 1
  74. 74. V +1 2
  75. 75. Final Results
  76. 76. Scalability 2011 2012 2013 2014
  77. 77. 9x response time
  78. 78. 3 years uptime
  79. 79. Go Hybrid! Thoughtfully >
  80. 80. Thank YOU! @majidfn
  81. 81. References Polyglot Persistence NoSQL Distilled

×