Database Cluster as a Service in OpenStack - Integrated, Scalable, Highly Available and Secure A webinar by Ivan Zoratti on 14.11.2013

1. Database
Cluster
as
a
Service
in
OpenStack
Integrated,
Scalable,
Highly
Available
and
Secure
Ivan
Zora>,
CTO
-­‐
14.11.2013
Thursday, 14 November 13

2. Who
is
Ivan
?
Thursday, 14 November 13

3. SkySQL
•Leading
provider
of
open
source
databases,
services
and
solutions
•Home
for
the
founders
and
the
original
developers
of
the
core
of
MySQL
•The
creators
of
MariaDB,
the
drop-­‐in,
innovative
replacement
of
MySQL
Thursday, 14 November 13

4. MySQL
,
MariaDB
and
Percona
Server
in
the
Cloud
Thursday, 14 November 13

5. The
Status
of
MySQL
MySQL
• From
Oracle,
acquired
with
Sun
in
2010
• Trademark,
code,
documentation,
bugs
are
available
at
*.mysql.com
• 5.6
GA
now
• 5.7
DMR
available
for
testing
MariaDB
• 5.5
GA:
branch
from
Oracle
with
add-­‐ons
from
Percona
and
other
key
contributors
Percona
• 5.6
GA:
branch
from
Oracle
with
add-­‐ons
from
MariaDB
and
other
key
contributors
• 10.0
Alpha:
fork
from
Oracle
with
add-­‐ons
and
new
storage
engines
5
Thursday, 14 November 13

6. More
in-­‐depth
features
• 5.6
MySQL
• Scales
up
to
48
CPU
threads
• Performance
Schema
• Memcached
plugin
for
InnoDB
• InnoDB
API
• InnoDB
Full
Text
Indexing
• Replication
with
Global
Transaction
ID
• HA
utilities
• Improved
DDL
online
operations
• Transportable
tablespaces
• Subquery
optimization
• Batched
key
access
and
multi-­‐range
•
•
•
read
Thread
pool
PAM
Authentication
Audit
plugin
• 5.5
MariaDB
• Subquery
optimization
• Batched
key
access
and
multi-­‐range
•
•
•
•
•
•
•
•
•
•
•
•
read
Index
condition
pushdown
and
many,
many
other
improvements
in
the
optimizer...
SphinxSE
and
ScaleDB
engines
Handler
Socket
Dynamic
and
virtual
columns
Group
Commit
for
binary
log
Microsecond
Support
GIS
operations
XtraDB
Engine
PAM
Authentication
Thread
pool
Audit
plugin
• 5.6
Percona
• Porting
of
the
MySQL
5.6
features
• XtraDB
5.6
• Per-­‐table/index/user/client
•
performance
counters
FlashCache
support
6
Thursday, 14 November 13

7. More
in-­‐depth
features
MySQL
• 5.7
• Improved
DDL
online
operations
• EXPLAIN
improvements
• Improved
triggers
• MySQL
Client
>
syslog
• Parallel
thread
replication
• EXPLAIN
in
JSON
format
• EXPLAIN
for
running
statements
• Subquery
optimization
• Loss-­‐less
MySQL
Replication
• Multi-­‐source
replication
MariaDB
• 10.0
• Includes
XtraDB
5.6
• Performance
schema
• Fusion
I/O
atomic
writes
• Further
optimizer
improvements
• Improved
DDL
online
operations
• Replication
with
Global
Transaction
ID
• Parallel
thread
replication
• Multi-­‐source
replication
• EXPLAIN
for
running
threads
and
logged
•
•
•
queries
Engine-­‐independent
table
statistics
Per-­‐engine
table
attributes
Per-­‐thread
memory
usage
• TokuDB,
Spider,
CASSANDRA
and
CONNECT
storage
engine
7
Thursday, 14 November 13

8. High
Availability
MySQL
MariaDB
Percona
• MySQL
Replication
with
HA
• MySQL
Replication
with
MHA
• MySQL
Replication
with
• DRBD
from
Linbit
• Shared
storage
with
Oracle
• MariaDB
Galera
Cluster
• Percona
XtraDB
Cluster
• Support
from
SkySQL
for:
•DRBD
•Shared
storage
• Support
for:
•DRBD
•Shared
storage
utilities
Enterprise
Linux
and
Pacemaker
Synchronous
Replication
with
Codership
Galera
Percona
Replication
Manager
Synchronous
Replication
with
Codership
Galera
8
Thursday, 14 November 13

9. Scalability
MySQL
• Improved
InnoDB
scalability
• Group
commit,
parallel
and
multi-­‐source
replication
in
5.7
• MySQL
Replication
for
read
scalability
• MySQL
Cluster
for
high
volume/small
transactions
MariaDB
• In
MariaDB
5.5:
• XtraDB
5.5
• Group
commit
• MySQL
Replication
for
read
scalability
• TokuDB
and
ScaleDB
engine
• Plus,
in
MariaDB
10:
• XtraDB
5.6
• Parallel
and
multi-­‐source
replication
• Spider
and
Cassandra
engine
• MariaDB
Galera
Cluster
for
Percona
• XtraDB
5.6
• Group
commit
• MySQL
Replication
for
read
scalability
• Percona
XtraDB
Cluster
for
synchronous
replication
synchronous
replication
• MySQL
Cluster
for
high
volume/small
transactions
9
Thursday, 14 November 13

10. Ease
of
Use
MySQL
• MySQL
Replication
Utilities
• Enterprise
Monitoring
• Online
Backup
• MySQL
Workbench
MariaDB
• MHA
• Enterprise
Monitor
• Xtrabackup
• Query
Editor
• MariaDB
Manager
Percona
• Percona
Replication
Manager
• Nagios
and
Cacti
plugins
• Xtrabackup
• Percona
Toolkit
10
Thursday, 14 November 13

11. MySQL
in
Rackspace
•Rackspace
Cloud
Database
•MySQL
Community
5.1
•Up
to
16GB
RAM,
150GB
storage
•Standard
Servers
•Up
to
8
vCPUs,
30GB
RAM,
1.2TB
storage
•BYODBMS
11
Thursday, 14 November 13

12. MySQL
in
HPCloud
•Manage
a
single
DB
instance
through
the
REST
API
•Standard
Servers
•Up
to
8
vCPUs,
30GB
RAM,
960GB
storage
•BYODBMS
•Marketplace
•GenieDB
•MariaDB
ScaleDB
12
Thursday, 14 November 13

13. MySQL
in
AWS
•Amazon
RDS
•MySQL
Community
5.1,
5.5
&
5.6
•Reserved
instances,
provisioned
IO
and
Flash
storage
• 32
vCPUs,
244GB
Memory,
10GB
network
•Automatic
backups,
Storage
replication,
MySQL
Replication
•Limited
tuning
•Standard
Servers
and
Marketplace
•Similar
instances
•MariaDB
on
CentOS,
Debian
and
Ubuntu
•GenieDB
13
Thursday, 14 November 13

14. MySQL
and
OpenStack
Thursday, 14 November 13

15. MySQL
[or
MariaDB,
or
Percona]
in
OpenStack
•Used
as
repository
-­‐
alternative
to
SQLite
•Increased
portability
and
availability
•High
Availability
is
the
key
factor:
•MySQL
Replication
•DRBD
•Galera
15
Thursday, 14 November 13

16. MySQL
Replication
Control
Network Compute
Node
Manager
HA
Manager
Database
Database
Database
Database
Database
Database
16
Thursday, 14 November 13

17. DRBD
Control
Network Compute
Database
Database
Database
Database
17
Thursday, 14 November 13

18. Galera
Cluster
Control
Database
Network Compute
Database
Database
Database
Database
Database
18
Thursday, 14 November 13

19. MySQL
as
a
Service
Thursday, 14 November 13

20. Automatic
Provisioning
•REST-­‐based
for
interoperation
•GUI-­‐assisted
for
end-­‐users
•From
server
to
cluster
20
Thursday, 14 November 13

21. The
false
promise
of
the
elasticity
•Distributed
MySQL
is
not
something
you
can
have
out
of
the
box
•Or
not
for
all
your
applications
Read-Write
Read-Only
Read-Only
M
1
binlog
LU
Database
relaylog
Thursday, 14 November 13
relaylog
relaylog
relaylog
2
3
4
SHARD
0-99
SHARD
100-199
SHARD
200-299
SHARD
300-399
LU
LU
LU
LU
Database
Database
Database
Database
21

22. DBaaS
vs
PaaS
vs.
Database
Database
22
Thursday, 14 November 13

23. Storage
•Backups
on
Swift
•Full
or
incremental
•Moving
tablespaces
in
Cinder
•For
high
availability
•For
backups
and
archiving
•For
maintenance
in01
Database
Database
in02
Database
in03
Database
in04
Database
Database
in05
Database
23
Thursday, 14 November 13

24. Networking
•Multiple
NICs
and
floating
IPs
•Public
IPs
•Local
proxies
vs
enhanced
connectors
Database
Database
Database
Database
Database
Database
24
Thursday, 14 November 13

25. Security
•LDAP
&
Keystone
•Integration
with
MySQL
LDAP
•LDAP
/
OpenLDAP
•MySQL
Security
Plugin
using
OpenLDAP
Keystone
Database
25
Thursday, 14 November 13

26. MariaDB
Cluster
in
OpenStack
Thursday, 14 November 13

27. What’s
new
with
MariaDB
MariaDB Manager
Configuration
& Provisioning
GUI
MariaDB Cluster
MaxScale
MHA
Galera
API
Monitor
MariaDB
Server(s)
27
Thursday, 14 November 13

28. SkySQL
MaxScale
Client
Client
Client
Client
Client
Client Applications
Frontend
Frontend
Frontend
Frontend
Frontend
Auth
Auth
Listener
MaxScale
Core
Server
Listener
Service
Listener
Service
Listener
Auth
Router
Service
Router
Server
Server
Server
Server
Router
Monitor
Backend
Backend
Backend
Monitor
Data Stores
28
Thursday, 14 November 13

29. Monitor/Management
Interface
Introducing
MariaDB
Manager
Monitor(Data
Provider
Monitor/Management
Interface
Monitor/Management
Interface
RESTful(API
Core
Agent
Agent
Agent
MariaDB(Galera(
Cluster
29
Thursday, 14 November 13

30. MariaDB
Galera
Cluster
-­‐
State
30
Thursday, 14 November 13

31. Provision
a
new
node
1
HTTP
POST
method
-­‐
Create
a
node
URI:
• .../restfulapi/system/systemid/node
• Parameters:
• name
• hostname
• publicip,
privateip,
port
• instanceid
• dbusername,
dbpassword
• ...
31
Thursday, 14 November 13

32. Provision
a
new
node
1
HTTP
POST
method
-­‐
Create
a
node
2
URI:
• .../restfulapi/system/systemid/nodeethod
-­‐
Run
the
command
HTTP
POST
m
• Parameters:
Connect
to
the
new
node
• name
URI:
• hostname
• publicip,
privateip,
port• .../restfulapi/command/connect
• instanceid
• Parameters:
• dbusername,
dbpassword• systemid
• ...
• nodeid
• rootpassword
32
Thursday, 14 November 13

33. Provision
a
new
node
1
HTTP
POST
method
-­‐
Create
a
node
2
URI:
• .../restfulapi/system/systemid/nodeethod
-­‐
Run
the
command
HTTP
POST
m
• Parameters:
3
Connect
to
the
new
node
• name
URI:
HTTP
POST
method
-­‐
Run
the
command
• hostname
• publicip,
privateip,
port• .../restfulapi/command/connect the
state
of
the
node
Probe
• instanceid
• Parameters:
URI:
dbusername,
dbpassword systemid
•
•
• .../restfulapi/command/probe
• ...
nodeid
•
• Parameters:
• rootpassword
• systemid
• nodeid
33
Thursday, 14 November 13

34. Provision
a
new
node
1
HTTP
POST
method
-­‐
Create
a
node
2
URI:
• .../restfulapi/system/systemid/nodeethod
-­‐
Run
the
command
HTTP
POST
m
• Parameters:
3
Connect
to
the
new
node
• name
URI:
HTTP
POST
method
-­‐
Run
the
command
• hostname
4
• publicip,
privateip,
port• .../restfulapi/command/connect the
state
of
the
node
Probe
• instanceid
• Parameters:
URI:
HTTP
POST
method
-­‐
Run
the
command
dbusername,
dbpassword systemid
•
•
• .../restfulapi/command/probe
• ...
Provision
the
node
nodeid
•
• Parameters:
URI:
• rootpassword
• systemid
• .../restfulapi/command/provision
• nodeid
• Parameters:
• systemid
• nodeid
34
Thursday, 14 November 13

35. Start
a
node
HTTP
POST
method
-­‐
Run
the
command
Start
the
node
URI:
• .../restfulapi/command/start
• Parameters:
• systemid
• nodeid
35
Thursday, 14 November 13

36. Retrieve
the
status
of
a
cluster
• .../restfulapi/system/systemid
36
Thursday, 14 November 13

37. Monitor
the
#
of
connections
of
a
node
• ../res^ulapi/system/systemid/
node/nodeid/monitor/
{monitorid}/data
37
Thursday, 14 November 13

38. What’s
coming
•MariaDB
Replication
and
MHA
•MariaDB
10
•MaxScale
integration
•Juju
charms
for
MariaDB
Cluster
and
MariaDB
Manager
•Puppet
&
Chef
manifests
and
recipes
•MariaDB
and
Trove
•Trove
for
MariaDB
Manager
38
Thursday, 14 November 13

39. For
More
Information...
•
MariaDB
Enterprise:
www.mariadb.org
www.mariadb.com
•
MariaDB
Manager
and
MaxScale:
http://downloads.skysql.com/
archives
•
Bugs:
http://bugs.skysql.com
39
Thursday, 14 November 13

40. Pictures
courtesy
of:
• www.pluralsight.com
Thank
You!
www.skysql.com
Thursday, 14 November 13
ivan@skysql.com
izoratti.blogspot.com
www.slideshare.net/izoratti