MySQL Cluster powers the subscriber databases of major communication services providers as well as next generation web, cloud, social and mobile applications. It is designed to deliver:
- Real-time, in-memory performance for both OLTP and analytics workloads
- Linear scale-out for both reads and writes
99.999% High Availability
- Transparent, cross-shard transactions and joins
- Update-Anywhere Geographic replication
- SQL or native NoSQL APIs
All that while still providing full ACID transactions.
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
What's new in MySQL Cluster 7.4 webinar charts
1. What’s new in MySQL Cluster 7.4
In-Memory Real-Time Performance, Web Scalability & 99.999% Availability
Andrew Morgan
Director MySQL Product Management
andrew.morgan@oracle.com
@andrewmorgan
www.clusterdb.com
9th April 2015
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved
2. Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 2
3. Focus on driving the business rather than on infrastructure
Driving new Database Requirements
In-Memory
Real-Time
Performance
Extreme
Read &
Write
Scalability
Rock Solid
Availability
Rapid
Service
Innovation
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 3
4. MySQL Cluster Overview
• Memory optimized tables with durability
• Predictable Low-Latency, Bounded Access Time
REAL-TIME
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE, READS +
WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations
99.999% AVAILABILITY
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++
SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools
LOW TCO
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 4
5. Who’s Using MySQL Cluster?
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 5
7. Oracle MySQL HA & Scaling Solutions
MySQL
Replication
MySQL
Fabric
Oracle VM
Template
Oracle
Clusterware
Solaris
Cluster
Windows
Cluster
DRBD
MySQL
Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode
Master +
Slaves
Master +
Slaves
Active/Pas
sive
Active/Passi
ve
Active/P
assive
Active/Pas
sive
Active/P
assive
Multi-
Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 7
8. Company Overview
Global telecommunications equipment company, focuses on
fixed, mobile, and converged networking hardware, IP
technologies, software, and services.
Application
MySQL Cluster CGE is at the heart of Alcatel-Lucent’s
Subscriber Data Manager to deliver converged IMS services
to mobile and fixed line users.
Why MySQL Cluster CGE?
• Delivers the performance, scalability and availability
required by this mission critical real-time application
• Real-time read & write access for tens of millions of
subscribers in a single system
• Always-on service: no offline maintenance window for
services users depend on (voice, SMS, email, Web, social
media…)
• Rapid delivery & low TCO
Alcatel-Lucent
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 8
9. Company Overview
PayPal allows any business or individual with an email
address to securely, conveniently and cost-effectively send
and receive payments online.
Application
PayPal built a cloud-based globally-distributed database
with 100 TB of user-related data based on MySQL Cluster.
“Must NOT lose data” system, delivering 99,999%
availability, transactional, with data available WW
anywhere in < 1 Sec.
Why MySQL Cluster?
“You can achieve high performance and availability
without giving up relational models and read
consistency.” Daniel Austin, Chief Architect, PayPal
PayPal
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 9
10. Company Overview
Playful Play created Latin America’s #1 Facebook game based
on "El Chavo del 8“.
Application
Playful Play selected MySQL Cluster CGE running on a public
cloud to power their gaming platform, providing:
• 45% improvement in performance
• 99.999% uptime
• 80% reduction in DBA overhead
• Local language support, 24x7
Why MySQL Cluster CGE?
“MySQL Cluster has allowed us to keep pace with the rapid
success of the game. Being the number one Facebook game
in Latin America is a big responsibility, and we rely on
Oracle’s products and MySQL support for continued growth.”
Ricardo Rocha, CEO, Playful Play
Playful Play
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 10
11. MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 11
12. MySQL Cluster Node (process) Types
MySQL Server
• SQL Access to data
• Round-robin app access
• Geographic replication
• Scale out for performance
Data Node
• Data storage & management
• Transparent sharding
• Synchronous replication
• Scale out for capacity/perf
Management Node
• Distribute configuration data
• Control Cluster membership
• Arbitrator in case of n/w
partitioning
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 12
13. Node Group
Data Node 1
Data Partitioning
• Data transparently sharded between Node Groups
• Each fragment active in one Data Node with synchronous replication to
2nd Data Node in same Node Group
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 13
Fragment 1
Fragment 2’
Data Node 2
Fragment 1’
Fragment 2
Node Group
Data Node 3
Fragment 3
Fragment 4’
Data Node 4
Fragment 3’
Fragment 4
14. Data Partitioning
• DBA chooses which part of
Primary Key to use as shard
key
• Fragment for each row
decided by hashing the
sharding key
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 14
User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
1773467253 photos xxx
7234782739 photos xxx
8235602099 reminders xxx
8437829249 location xxx
Shard Key
15. MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 15
16. MySQL Cluster Scaling
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 16
17. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 17
18. MySQL Cluster HA
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 18
19. On-line Operations
• Scale the cluster
– Add all nodes
– Remove MySQL Server nodes
• Repartition tables
• Upgrade / patch servers & OS
• Upgrade / patch MySQL Cluster
• Back-Up
• Evolve the schema on-line, in real-time
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 19
20. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 20
21. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 21
22. MySQL Cluster 7.2: 1.2 Billion UPDATEs per Minute
• NoSQL C++ API,
flexaSynch benchmark
• 30 x Intel E5-2600 Intel
Servers, 2 socket, 64GB
• ACID Transactions, with
Synchronous Replication
http://www.mysql.com/why-
mysql/white-papers/mysql-cluster-
benchmarks-1-billion-writes-per-minute/0
5
10
15
20
25
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
MillionsofUPDATEsperSecond
MySQL Cluster Data Nodes
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 22
23. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 23
24. MySQL Cluster Auto-Installer
• Fast configuration
• Auto-discovery
• Workload optimized
• Repeatable best practices
Specify
Workload
Auto-
Discover
Define
TopologyDeploy
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 24
25. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 25
26. MySQL Cluster Manager
1. Download MCM/Cluster package from edelivery.oracle.com:
2. Unzip
3. Run agent, define, create & start Cluster!
$> binmcmd --bootstrap
MySQL Cluster Manager 1.1.2 started
Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862
Configuring default cluster 'mycluster'...
Starting default cluster 'mycluster'...
Cluster 'mycluster' started successfully
ndb_mgmd NOVA:1186
ndbd NOVA
ndbd NOVA
mysqld NOVA:3306
mysqld NOVA:3307
ndbapi *
Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 26
27. 9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 27
28. NoSQL Access to MySQL Cluster data
Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps
JPA
Cluster JPA
PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached
MySQL JNI Node.JS mod_ndb ndb_eng
NDB API (C++)
MySQL Cluster Data Nodes
29. SQL
• Industry standard
• Joins & Complex queries
• Relational model
Memcached
• Simple to use API
• Key/value
• Drivers for many languages
Mod-ndb
• REST
• Html
• Plugin for Apache
ClusterJ
• Simple to Use Java API
• Web & telco
• Object Relational Mapping
• Native & fast access to data
ClusterJPA
• OpenJPA plugin
• Standards defined ORM
• Cross table Joins
JavaScript/Node.js
• Native JavaScript: client to
DB
• Blazing fast asynchronous
throughput
Choosing the right application API
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 29
30. Schema-Free apps
• Rapid application evolution
– New types of data constantly
added
– No time to get schema extended
– Missing skills to extend schema
– Initially roll out to just a few users
– Constantly adding to live system
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 30
31. Cluster & Memcached – Schema-Free
<town:maidenhead,SL6>
key value
<town:maidenhead,SL6>
key value
Key Value
town:maidenhead SL6
generic table
Application view
SQL view
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 31
32. Cluster & Memcached - Configured Schema
<town:maidenhead,SL6>
prefix key value
<town:maidenhead,SL6>
key value
Prefix Table Key-col Val-col policy
town: map.zip town code cluster
Config tables
town ... code ...
maidenhead ... SL6 ...
map.zip
Application view
SQL view
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 32
33. Node.js NoSQL API • Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app & DB
– Storing and retrieving JavaScript objects
directly in MySQL Cluster
– Eliminate SQL transformation
• Implemented as a module for node.js
– Integrates Cluster API library within the web
app
• Couple high performance, distributed apps,
with high performance distributed database
• Optionally routes through MySQL Server
– Use with InnoDB
V8 JavaScript Engine
MySQL Cluster Node.js Module
MySQL Cluster Data Nodes
Clients
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 33
34. NoSQL API for Node.js & FKs
FKs enforced on all APIs:
{ message: 'Error',
sqlstate: '23000',
ndb_error: null,
cause:
{message: 'Foreign key constraint violated: No parent row found [255]',
sqlstate: '23000',
ndb_error:
{ message: 'Foreign key constraint violated: No parent row found',
code: 255,
classification: 'ConstraintViolation',
handler_error_code: 151,
status: 'PermanentError' },
cause: null } }
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 34
35. MySQL Cluster 7.4 GA
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 35
36. MySQL Cluster 7.4 GA
• 200 Million NoSQL
Reads/Sec
• 2.5M SQL Ops/Sec
• 50% Faster Reads
• 40% Faster Mixed
Performance
• Active-Active
Geographic
Redundancy
• Conflict
Detection/Resolution
Active-Active
• 5X Faster
Maintenance Ops
• Detailed Reporting
Management
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 36
37. • Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed
searches
• MySQL Cluster 7.4 FlexAsych
– 200M NoSQL Reads/Second
16/04/2015 37
MySQL Cluster 7.4 NoSQL Performance
200 Million NoSQL Reads/Second
Copyright 2015, oracle and/or its affiliates. All rights reserved
-
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Readspersecond
Data Nodes
FlexAsync Reads
38. • Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed
searches
• MySQL Cluster 7.4 DBT2 BM
– 2.5M SQL Statements/Second
16/04/2015 38
MySQL Cluster 7.4 SQL Performance
2.5M SQL Statements/Second
Copyright 2015, oracle and/or its affiliates. All rights reserved
-
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
2 4 6 8 10 12 14 16
SQLStatements/sec
Data Nodes
DBT2 SQL Statements per Second
40. • Asynchronous replication between
MySQL Clusters
• Active-Active
– Update anywhere
– Conflict detection
• Application notified through exception tables
• Can opt to have conflicts resolved
automatically
– Auto-conflict-resolution
• Conflicting transaction and dependent ones
are rolled-back
• No changes to application schema
Active-Active Geo-Replication
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 40
41. What is a conflict?
John.balance==$100
John.balance-=$40
John.balance==$60
John.balance==$200
John.balance==$100
John.balance+= $100
John.balance==$200
John.balance==$60
Spend $40 Add $100
$60
$200
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 41
42. • NDB$EPOCH2 and
NDB$EPOCH2_TRANS introduced
• Detects conflicting
inserts/updates/deletes/reads
• Entire transactions (and dependent
ones) rolled back
• Rolling back of transactions that
read conflicted data
• Improved NDB Exceptions table
format
– Non-PK columns, operation type,
transaction id, before and after values
• Online conflict role change
9th April 2015 42
Handling of Conflicts – Extensions in MySQL Cluster 7.4
Copyright 2015, oracle and/or its affiliates. All rights reserved
43. • Primary stores logical timestamp (GCI)
against updated row
– Window for conflict opens
• GCI replicated with updated row to
Secondary
• The same row and GCI is replicated back
(reflected) from Secondary to Primary
after it has been applied
– Closing window for conflict
• Primary checks every event originating
from the Secondary to ensure it isn’t for a
‘conflictable’ row
Detecting Conflicts - Reflected GCI
John.balance==$100
John.balance-=$40
John.balance==$60
John.balance==$200
John.balance==$100
John.balance+= $100
John.balance==$200
John.balance==$60
Spend
$40
Add $100
$60
$200
Primary Secondary
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 43
44. How to Use Conflict Detection/Resolution
Decide which
tables need
protecting
For each table,
specify what to
do on conflicts
Just record in
exception table
Application or
DBA acts on
content
Rollback the
conflicting row
Rollback the
conflicting
transaction
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 44
45. MySQL Cluster 7.4 Restart Improvements
• Duration of long-running maintenance activities dominated by Data Node
restart times
• MySQL Cluster 7.4 = 5.5X faster restarts
• Benefits both nodal and rolling restarts
– Upgrades, add-node,…
• Benefits both SQL and NoSQL APIs
• Benefits both ”manual” and MySQL Cluster Manager operations
• Achieve 5x as much during a single maintenance window
Make Data Node Restarts Fast!
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 45
46. MySQL Cluster 7.4 Restart Improvements
• Verbose logging
– Task start/completion
– Data volumes
– Parallelism & Wait times
• NDBINFO for recent node restarts
• More documentation of stages
• Goal: Make analysis of a slow restart possible
– Determine cause; Detect patterns; Understand the impact of indexes, local
checkpoints etc.
Observability improvements
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 46
47. 2014-11-12 16:30:47 [ndbd] INFO -- Start phase 0 completed
2014-11-12 16:30:47 [ndbd] INFO -- Phase 0 has made some file system initialisations
2014-11-12 16:30:47 [ndbd] INFO -- Starting QMGR phase 1
2014-11-12 16:30:47 [ndbd] INFO -- DIH reported initial start, now starting the Node Inclusion Protocol
2014-11-12 16:30:50 [ndbd] INFO -- findNeighbours from: 2220 old (left: 65535 right: 65535) new (3 1)
2014-11-12 16:30:50 [ndbd] INFO -- Include node protocol completed, phase 1 in QMGR completed
2014-11-12 16:30:50 [ndbd] INFO -- Start phase 1 completed
2014-11-12 16:30:50 [ndbd] INFO -- Phase 1 initialised some variables and included node in cluster, locked memory if configured to do so
2014-11-12 16:30:50 [ndbd] INFO -- Asking master node to accept our start (nodeId = 1 is master), GCI = 0
2014-11-12 16:30:50 [ndbd] INFO -- findNeighbours from: 2132 old (left: 3 right: 1) new (3 2)
2014-11-12 16:30:50 [ndbd] INFO -- NDBCNTR master accepted us into cluster, start NDB start phase 1
2014-11-12 16:30:50 [ndbd] INFO -- We are performing initial start of cluster
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Starting REDO log initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Starting REDO log initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Starting REDO log initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Starting REDO log initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Completed REDO log initialisation of logPart = 0
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 3: Completed REDO initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Completed REDO log initialisation of logPart = 0
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 1: Completed REDO initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Completed REDO log initialisation of logPart = 0
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 4: Completed REDO initialisation
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Completed REDO log initialisation of logPart = 0
2014-11-12 16:30:50 [ndbd] INFO -- LDM instance 2: Completed REDO initialisation
2014-11-12 16:30:50 [ndbd] INFO -- Schema file initialisation Starting
2014-11-12 16:30:50 [ndbd] INFO -- Schema file initialisation Completed
2014-11-12 16:30:51 [MgmtSrvr] INFO -- Node 1: Make On-line Database recoverable by waiting for LCP Starting, LCP id = 1
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Local checkpoint 1 started. Keep GCI = 1 oldest restorable GCI = 1
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 1: Completed LCP, num fragments = 4 num records = 528, num bytes = 16976
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 2: Completed LCP, num fragments = 4 num records = 471, num bytes = 15152
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 3: Completed LCP, num fragments = 4 num records = 536, num bytes = 17232
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 2: LDM instance 4: Completed LCP, num fragments = 4 num records = 513, num bytes = 16496
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 2: Completed LCP, num fragments = 4 num records = 471, num bytes = 15152
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 3: Completed LCP, num fragments = 4 num records = 536, num bytes = 17232
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 1: Completed LCP, num fragments = 4 num records = 528, num bytes = 16976
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: LDM instance 4: Completed LCP, num fragments = 4 num records = 513, num bytes = 16496
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Local checkpoint 1 completed
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 1: Make On-line Database recoverable by waiting for LCP Completed, LCP id = 1
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 5 completed (initial start)
…
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 6 completed (initial start)
…
2014-11-12 16:30:52 [MgmtSrvr] INFO -- Node 4: Start phase 7 completed (initial start)
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 47
48. Restart Times
• Restarting data node with locally checkpointed data
– Major improvement
• Restarting data node which must recover data from peer
– Major improvement
– Further speedups to come in 7.4.X (greater parallelization)
• Upgrade/rolling restarts
– Major improvement
• Cluster shutdown and restart
– Minor improvement
What operations benefit?
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 48
49. MySQL Cluster 7.4 Restart Improvements
• Faster Local Checkpoints
– Parallel
– Tuning & Adaptive throttling
• Parallel operations
– Adaptive to avoid starving remaining system of resources
• Feedback loop
– Determine Cluster 7.3 baseline for your environment
– Experiment with Cluster 7.4 settings
– Consult Oracle with results to establish improvements to be made in configuration
Speedups
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 49
50. MySQL Cluster 7.4 – Enhanced memory reporting
• ndbinfo.memory_per_fragment memory usage information for
each fragment replica, for each table and index
• Allocated memory and how much of that is actually in use.
• Exposes
– Fragmentation of fixed and var-sized fragment pages
– Accurate Data and Index Memory use
– Comparison of Primary and Backup fragment usage
– Partitioning effectiveness
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 50
54. Enhanced Memory Reporting
mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT, species VARCHAR(20) DEFAULT "Human",
PRIMARY KEY(id, species)) engine=ndb PARTITION BY KEY(species);
// Add some data
mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes,
fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from ndbinfo.memory_per_fragment
where fq_name like '%simples%';
+------+------+-------------+------------+------------+
| node | frag | alloc_bytes | free_bytes | spare_rows |
+------+------+-------------+------------+------------+
| 1 | 0 | 0 | 0 | 0 |
| 1 | 2 | 196608 | 11732 | 419 |
| 2 | 0 | 0 | 0 | 0 |
| 2 | 2 | 196608 | 11732 | 419 |
| 3 | 1 | 0 | 0 | 0 |
| 3 | 3 | 0 | 0 | 0 |
| 4 | 1 | 0 | 0 | 0 |
| 4 | 3 | 0 | 0 | 0 |
+------+------+-------------+------------+------------+
Check how well partitioned/sharded a table is
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 54
55. MySQL Cluster 7.4 – Enhanced activity reporting
• ndbinfo.operations_per_fragment activity counters for each
fragment replica, for each table and index
• PK & scan access – requests, bytes, rows…
• Exposes
– How traffic maps to tables and indices
– Query execution, use of indexes etc.
– LDM and node imbalances
– Hotspots and scan overloads
16/04/2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 55
57. When to Consider MySQL Cluster
Scalability demands
Sharding for write performance?
Latency demands
Cost of each millisecond?
Uptime requirements
Cost per minute of downtime?
Failure versus maintenance?
Application agility
Developer languages and frameworks?
SQL or NoSQL?
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 57
58. General Usage Considerations
• MySQL Cluster is designed for
– Short transactions
– Many parallel transactions
• Utilize simple access patterns for high running transactions
– Use efficient scans and batching interfaces
– AQL gives huge performance in JOIN operations
• Storage engine configurable for each table…InnoDB or NDB
MySQL Cluster Evaluation Guide
http://mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 58
60. Enhancing DevOps Agility,
Reducing Downtime
Automated Management
• Start / Stop node or
whole cluster
• On-Line Scaling
• On-Line Reconfiguration
• On-Line Upgrades
• On-Line Backup &
Restore
• Import Running Cluster
Self-Healing
• Node monitoring
• Auto-recovery extended
to SQL + mgmt nodes
HA Operations
• Cluster-wide
configuration
consistency
• Persistent
configurations
• HA Agents
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 60
61. Without MySQL Cluster Manager
• 1 x preliminary check of cluster state
• 8 x ssh commands per server
• 8 x per-process stop commands
• 4 x scp of config files (2 x mgmd & 2 x mysqld)
• 8 x per-process start commands
• 8 x checks for started and re-joined processes
• 8 x process completion verifications
• 1 x verify completion of the whole cluster.
• Excludes manual editing of each configuration file.
• Total: 46 commands
– 2.5 hours of attended operation
With MySQL Cluster Manager
mcm> upgrade cluster
--package=7.4 mycluster;
• Total: 1 Command -
– Unattended Operation
Software Upgrade – example of benefits of MCM?
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 61
62. MySQL Cluster Manager 1.3.3 GA
“Unmanaged” production Cluster
mcm> create cluster --import
mcm> import config [--dryrun]
mcm> import cluster[--dryrun]
Cluster now managed by MCM
Import a running Cluster into MCM
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 62
63. Oracle Premier Support for MySQL
• Straight from the Source
• Largest Team of MySQL Experts
• Backed by MySQL Developers
• Forward Compatible Hot Fixes
• MySQL Maintenance Releases
• MySQL Support in 29 Languages
• 24/7/365
• Unlimited Incidents
• Knowledge Base
• MySQL Consultative Support
Rely on The Experts - Get Unique Benefits
"The MySQL support service has been essential
in helping us with troubleshooting and
providing recommendations for the production
cluster, Thanks."
-- Carlos Morales – Playfulplay.com
Only From
Oracle
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 63
64. Oracle University MySQL Training Services
Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications
“Training and team skill
have the most significant impact on overall performance of
technology and success of technology projects.” - IDC, 2013
Premier Support customers eligible to
save 20% on learning credits.
Benefits
Expert-led training to support your MySQL learning needs
Flexibility to train in the classroom or online
Hands-on experience to gain real world experience
Key skills needed for database administrators and developers
• MySQL for Beginners
MySQL for Database Administrators
MySQL Performance Tuning
MySQL Cluster – NEW - Register Your Interest!
MySQL and PHP - Developing Dynamic Web Applications
MySQL for Developers
MySQL Developer Techniques
MySQL 5.6 Database Administrator
MySQL 5.6 Developer
To find out more about available MySQL Training & Certification
offerings, go to: education.oracle.com/mysql
Top Courses for Administrators and Developers
Top Certifications
RECENTLY RELEASED
!!ALL NEW!! MySQL Cluster Training
To Register your interest to influence the
schedule on this newly released course – go to
education.oracle.com/mysql and click on the
MySQL Cluster Course
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 64
65. Next Steps
Learn More
• www.mysql.com/cluster
• Authentic MySQL Curriculum: http://oracle.com/education/mysql
Try it Out
• dev.mysql.com/downloads/cluster/
Let us know what you think
• clusterdb.com
• @clusterdb
• forums.mysql.com/list.php?25
9th April 2015 Copyright 2015, oracle and/or its affiliates. All rights reserved 65