Thu 1100 duncan_john_color

<Insert Picture Here>

MySQL Cluster with and without SQL
John David Duncan
Senior Software Engineer, Oracle Corp.
2

Program Agenda


• Overview of MySQL Cluster
• SQL interface
• NoSQL interfaces
– mod_ndb
– ClusterJ
– Memcache

3

The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any features
or functionality described for Oracle’s products
remains at the sole discretion of Oracle.


MySQL Cluster Overview
ACID Compliant Relational Database
SQL & NoSQL interfaces

Write-Scalable & Real-Time
Distributed, auto-partitioning (sharding), multi-master

99.999% Availability
Shared-nothing, integrated clustering & sub-second
recovery, local & geographic replication, on-line
operations

Low TCO
Open-source, management & monitoring tools, scale-out
on commodity hardware


The Basics
API Nodes

Node Group 1 Node Group 2
NDB Data Nodes


Geographic Replication

Synchronous
Cluster 1 Cluster 2 replication

Asynchronous
replication

MyISAM MyISAM InnoDB


MySQL Cluster – Users & Applications
HA, Transactional Services: Web & Telecoms

• Web
• User profile management
• Session stores
• eCommerce
• On-Line Gaming
• Application Servers

• Telecoms
• Subscriber Databases (HLR/HSS)
• Service Delivery Platforms
• VoIP, IPTV & VoD
• Mobile Content Delivery
• On-Line app stores and portals
• IP Management
• Payment Gateways
http://www.mysql.com/customers/cluster/


SQL interface <Insert Picture Here>



The long road to SQL JOIN performance

• Initial integration (2004)
– Move the data to the query
– ... one row at a time
– ... one loop iteration in a nested-loop join = 1 network trip
• 2005 - 2009
– Improve efficiency ...
– within the existing framework of the MySQL optimizer
– e.g. Batch Key Access, push-down filters
• MySQL Cluster 7.2 (2011)
– Adaptive Query Localization
– Algorithmic changes to optimizer
– Push the majority of the query to the data nodes

10

Results: Adaptive Query Localization
in MySQL Cluster 7.2
mysql> SELECT COUNT(*)
FROM residents, postcodes, towns
WHERE residents.postcode=postcodes.postcode
AND postcodes.town=towns.town
AND towns.county="Berkshire";

+----------+
| COUNT(*) |
+----------+
| 40001 |
+----------+

Before: 48.68 sec • After: 2.02 sec


NoSQL interfaces <Insert Picture Here>

mod_ndb Memcache
ClusterJ



SQL and NoSQL Together

Schema creation
Reporting
Analytics Node Group 1

Real-Time Operations

Node Group 2


mod_ndb <Insert Picture Here>


mod_ndb

http://mod-ndb.googlecode.com/
• First released March 2007
• Use REST and JSON to manage data
stored in MySQL Cluster
Apache
• Open source project based on C++
NDBAPI
NDB API

Data Data
Node Node


mod_ndb

httpd.conf
<Location /ndb/app/car>
SELECT * from cars Apache
WHERE PRIMARY KEY = $id;
Format JSON NDB API
</Location>

Data Data
Node Node


mod_ndb

Client GET /ndb/app/car?id=371 HTTP/1.1

Apache

NDB API

Data Data
Node Node


mod_ndb

Client GET /ndb/app/car?id=371 HTTP/1.1

HTTP/1.1 200 OK
ETag: 700847d8b5b0901a2f2451efc4c4
Content-type: application/json Apache
Content-length: 90
NDB API
{ "car_id" : 371,
"tag" : "807AHC",
"state" : "OR", Data Data
"make" : "Dodge", Node Node
"year" : 1984
}

The view from Javascript

Browser reply = XMLHttpRequest.get();

Apache

NDB API

my_car = JSON.parse(reply);
Data Data
Node Node


mod_ndb

http://mod-ndb.googlecode.com/
• Very small user community
• Not officially supported by a vendor
• Not as widely functional as Cluster/J
• Not as fast as NDB+Memcache
• Moves an unusually large portion of application logic into httpd.conf
• Could use a major upgrade (maybe later this year?)

20

ClusterJ <Insert Picture Here>


ClusterJ

• High Performance, Light Weight, Easy to Use Direct
Connection
– In the style of Hibernate / JPA / JDO
• Shared Data storage with:
– MySQL server
– Native C++ applications
– Other ClusterJ applications
• Domain Object Model DataMapper pattern
– Data is represented as domain objects
– Domain objects are separate from business logic
– Domain objects are mapped to database tables
• Does not support relationships
– Look at JDO / JPA for these modeling patterns

22

Domain Object Model Mapping

• Tables map to Persistent Interfaces / Classes
• Columns map to Persistent Properties
– column names default to property name
• Rows map to Persistent Instances
• Annotations on Interfaces / Classes customize
mappings
• User chooses to write:
– User interface (ClusterJ then generates implementation class)
– Persistent class (ClusterJ provides base implementation class)


ClusterJ – Generated Class
@PersistenceCapable(table="employee")
public interface Employee {

long getId();

void setId(int id);

@Column(name="full_name")

String getName();

void setName(String value);

int getSalary();

void setSalary(int value);

Integer getAge();

void setAge(Integer value);
}


ClusterJ – Dynamic Object
public class Employee

extends DynamicObject {

public String table() {

return "employee";

long getId() {

return (Long)get(0);

}

void setId(long value) {

set(0, value);

}
...
// other fields and behavior
}


Numeric Column Mapping
• Java boolean, Boolean • Java long, Long
– BIT(1) – BIT(1) to BIT(64)
• Java byte, Byte – BIGINT
– BIT(1) to BIT(8) – BIGUNSIGNED
– TINYINT • Java float, Float
• Java short, Short – FLOAT
– BIT(1) to BIT(16) • Java double, Double
– SMALLINT – DOUBLE
– YEAR • Java BigDecimal
• Java int, Integer – NUMERIC
– BIT(1) to BIT(32) – DECIMAL
– INT • Java BigInteger
– NUMERIC
– DECIMAL

Date Column Mapping

• Java util Date
– DATETIME
– TIMESTAMP
– TIME
– DATE
• Java sql Date
– DATE
• Java sql Time
– TIME
• Java sql Timestamp
– DATETIME
– TIMESTAMP


Variable Size Column Mapping

• Java String
– CHAR
– VARCHAR
– TEXT
• Java byte[ ]
– BINARY
– VARBINARY
– BLOB


ClusterJ Features

• Character Set Translation (all MySQL charsets)
• Automatic detection of primary keys, indexes
• Compound Primary Keys
• Ordered (btree) indexes
• Unique (hash) indexes
• Automatic use of partition key
• Multi-threaded applications


ClusterJ Limitations
• No Relationships
– primitive types only
• No Multi-table inheritance
– single table per persistent interface or class
• No joins in queries
– column comparisons and boolean operators
• No Table creation
– user needs to create tables and indexes
• No Lazy Loading
– entire record is loaded at one time, including LOBs


ClusterJ Interfaces

• SessionFactory
– Instance per connection to cluster
• Session
– Instance per "user"
– persist(), remove(), update(), write()
– Find by ID
• Transaction
– Instance per Session
– begin(), commit(), rollback()
• Query
– Multiple instances per Session


ClusterJ User View
Domain
Object Domain
Domain Object
Domain Object Domain
Object Domain Object Domain
Domain Object Object Domain
Object Domain Domain Object
Object Object Domain
Object

Session and
Transaction
Session and
Session and
Transaction
Transaction

SessionFactory Configuration Properties


Example
Session session;
void getSession() {

session = sessionFactory.getSession();
}
Employee createEmployee(long id, String name,

int salary, int age) {

Employee employee =

session.newInstance(Employee.class);

employee.setId(id);

employee.setName(name);

employee.setSalary(salary);

employee.setAge(age);

session.persist(employee);

return employee;
}


Example
Transaction transaction;

void getTransaction() {

transaction = session.getTransaction();
}

void createEmployees() {

getTransaction();

transaction.begin();

createEmployee(1, "Amos", 10000, 44);

createEmployee(2, "Barbara", 14000, 48);

createEmployee(3, "Chuck", 78000, 61);

createEmployee(4, "Dave", 3000, 22);

transaction.commit();
}


Query

• Builder pattern
• Similar to JPA criteria query
• Compare column values to parameters:
– equal, lessEqual, greaterEqual, lessThan, greaterThan, in
– comparison with null
• Combine terms using boolean operators:
– or, and, not
• Execution is optimized to use indexes
– primary or unique key lookup
– ordered scan for complete or partial keys
– table scan if no index can be used

35

Query Example
QueryDomainType qemp =
builder.createQueryDefinition(Employee.class) ;

Predicate geAge = qemp.get("age")

.greaterEqual(qemp.param("ageFloor"));
Predicate leSalary = qemp.get("salary")

.lessEqual(qemp.param("salaryCap"));

qemp.where(geAge.and(leSalary));

Query query = session.createQuery(qemp) ;

query.setParameter("ageFloor",33);
query.setParameter("salaryCap", 44000);

List<Employee> results = query.getResultList() ;


Performance


Memcache API <Insert Picture Here>


Memcached Overview:
Two levels of hashing

memcached
httpd

hash key
PHP/Perl ve r memcached
ic k ser to find data
ke y to p
Memcache hash
friends:12389 memcached
memcache key


Cache hit

httpd

hash key
ke y to p
Memcache hash
friends:12389 VALUE friends:12389 0 31rn
101, 11009, 11150, 55881, 77798 rn


Cache miss: fetch from DB

httpd

hash key
memcache
ke y to p
hash
Memcache F OUND
NOT
mysql
SELECT
friend_id
FROM u MySQL
ser_frien
WHERE ds
user_id = Slave
?


Expected Latency & Throughput

httpd c.
t ions/se
o f opera
10,000s nd trip
u
PHP/Perl ~2 00 µs ro memcached

memcache

mysql
1,000s of
operatio
~ 2 ms ro ns/sec. MySQL
und trip Slave


Goals
• Access stored data directly from memcache clients

– Memcached perspective:
• MySQL Cluster is a write-scalable, replicated data store
– with reliable in-memory storage,
– plus on-disk storage when data is too big for
memory.
– MySQL Cluster perspective:
• memcache is a high performance API
– providing easy access to in-memory data,
– plus an extra layer of caching when data is on disk.


Goals
• Support existing schemas and all MySQL data types
• Cache MySQL Cluster data inside memcached when
desired
– with automatic cache management
– and flexibility to fine-tune (or disable) the cache policies
• Support the whole memcache protocol
• Achieve superior performance
– latency as expected from memcached
– throughput as expected from memcached


A Key Prefix

user:1248

the prefix the database key


Standard Tables in ndbmemcache
• meta
– stores configuration schema version (for upgrade
compatibility); consider it to be read-only
• ndb_clusters
• containers
– existing tables where data is stored
• cache_policies
– rules describing how it can be accessed
• key_prefixes
• memcache_server_roles
• last_memcached_signon
• demo_table

A key-prefix mapping

Memcache
Cache
key Cluster Container
Policy
prefix


A memcache server role

key Con- Cache
Cluster
prefix tainer Policy

Server
Role key Con- Cache
Cluster
ID prefix tainer Policy

key Con- Cache
Cluster

key Con- Cache
Cluster


Measured Latency memcachetest -t 2 -M 7000 -c 25000


Measured Throughput memslap


Limitations of Memcache API

• The size of stored values is limited to the MySQL
Cluster row size (without BLOBs)
– This is about 13KB (up from 8KB in 7.1)


Program Review

Overview of MySQL Cluster
Highly available
Write-scalable
SQL interface
Redesigned in 7.2
NoSQL interfaces
mod_ndb, ClusterJ, Memcache
plus: C++ NDBAPI, LDAP, etc.


Thu 1100 duncan_john_color

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Thu 1100 duncan_john_color

Similar to Thu 1100 duncan_john_color (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

Thu 1100 duncan_john_color