MySQL an be used as a NoSQL JSON Document Store as well as its well known ability as a SQL Relational Data Base. This presentation covers why you would want to use NoSQL and JSON and how to combine it what the relational data you already have

1. MySQL
Document Store -
A NoSQL JSON
Document Database
Dave Stokes
@stoker
david.stokes@oracle.com
Elephantdolphin.blogger.com
OpensourceDBA.wordpress.com

2. MySQL
Without the SQL -
Oh My!
Dave Stokes
@stoker
david.stokes@oracle.com
Elephantdolphin.blogger.com
OpensourceDBA.wordpress.com

3. 3

4. Safe Harbor Agreement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY
NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO
DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE
RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT,
RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED
FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.
4

5. MySQL Community Edition
5

6. NoSQL & SQL
6
Together!
MySQL
Document
Store

7. Relational Databases
7

8. Relational Databases
● Data Integrity
○ Normalization
○ constraints (foreign keys, ...)
● Atomicity, Consistency, Isolation, Durability
○ ACID compliant
○ transactions
● SQL
○ powerful query language 8

9. Relational Databases
● Need to set up tables BEFORE use
● Relations, indexes, data normalization, query optimizations
● Hard to change on the fly
● Need a DBA or someone who has DBA skills
9

10. NoSQL or Document Store
10

11. NoSQL or Document Store
● Schemaless
○ No schema design, no normalization, no foreign keys, no data types, …
○ Very quick initial development
● Flexible data structure
○ Embedded arrays or objects
○ Valid solution when natural data can not be modelized optimally into a
relational model
○ Objects persistence without the use of any ORM - *mapping object-
oriented*
11

12. NoSQL or Document Store
● JSON
● close to frontend
● native in JS
● easy to learn
12

13. How DBAs see data as opposed to how Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
13

14. What if there was a way to provide both
SQL and NoSQL on one stable platform that
has proven stability on well know
technology with a large Community and a
diverse ecosystem ?
With the MySQL Document
Store it is now an option!
14

15. A Solution for all
Developers:
schemaless
★ rapid prototyping
& simpler APIs
★ document model
★ transactions
Operations:
★ performance
management/visibility
★ robust replication,
backup, restore
★ comprehensive tooling
ecosystem
★ simpler application
schema upgrades 15
Business Owner:
★ don't lose my data ==
ACID trx
★ capture all my data =
extensible/schemaless
★ product on
schedule/time to
market = rapid
development

16. Built on the MySQL JSON Data type and Proven MySQL Server Technology 16
★ Provides a schema flexible JSON Document Store
★ No SQL required
★ No need to define all possible attributes, tables,
etc.
★ Uses new X DevAPI
★ Can leverage generated column to extract JSON
values into materialized columns that can be
indexed for fast SQL searches.

17. Built on the MySQL JSON Data type and Proven MySQL Server Technology 17
★ Document can be ~1GB
○ It's a column in a row of a table
★ Allows use of modern programming styles
○ No more embedded strings of SQL in your code
○ Easy to read
★ Also works with relational Tables
★ Proven MySQL Technology

18. ★ C++
★ Java
★ .Net
★ Node.js
★ JavaScript
★ Python
★ PHP
○ Working with other Communities to help them supporting it too 18
Connectors for

19. ★ Command Completion
★ Python, JavaScripts & SQL modes
★ Admin functions
★ New Util object
★ A new high-level session concept that can scale from single MySQL
Server to a multiple server environment
19
New MySQL Shell

20. ★ Non-blocking, asynchronous calls follow common language patterns
★ Supports CRUD operations
20
New Model

21. 21
X Protocol built on Google Protobufs

22. 22
Architecture of both Old and New Protocols

23. 23
How Your Application will work with InnoDB Cluster

24. But what does this look like in PHP?? 24

25. JavaScript 25
// Connecting to MySQL Server and working with a Collection
var mysqlx = require('mysqlx');
// Connect to server
var mySession = mysqlx.getSession( {
host: 'localhost', port: 33060,
user: 'user', password: 'password'} );
var myDb = mySession.getSchema('test');
// Create a new collection 'my_collection'
var myColl = myDb.createCollection('my_collection');
// Insert documents
myColl.add({_id: '1', name: 'Sakila', age: 15}).execute();
myColl.add({_id: '2', name: 'Susanne', age: 24}).execute();
myColl.add({_id: '3', name: 'User', age: 39}).execute();
// Find a document
var docs = myColl.find('name like :param1 AND age < :param2').limit(1).
bind('param1','S%').bind('param2',20).execute();
// Print document
print(docs.fetchOne());
// Drop the collection
myDb.dropCollection('my_collection');

26. Python 26
# Connecting to MySQL Server and working with a Collection
from mysqlsh import mysqlx
# Connect to server
mySession = mysqlx.get_session( {
'host': 'localhost', 'port': 33060,
'user': 'user', 'password': 'password'} )
myDb = mySession.get_schema('test')
# Create a new collection 'my_collection'
myColl = myDb.create_collection('my_collection')
# Insert documents
myColl.add({'_id': '1', 'name': 'Sakila', 'age': 15}).execute()
myColl.add({'_id': '2', 'name': 'Susanne', 'age': 24}).execute()
myColl.add({'_id': '3', 'name': 'User', 'age': 39}).execute()
# Find a document
docs = myColl.find('name like :param1 AND age < :param2')
.limit(1)
.bind('param1','S%')
.bind('param2',20)
.execute()
# Print document
doc = docs.fetch_one()
print doc

27. Node.JS 27
// Connecting to MySQL Server and working with a Collection
var mysqlx = require('@mysql/xdevapi');
var db;
// Connect to server
mysqlx
.getSession({
user: 'user',
password: 'password',
host: 'localhost',
port: '33060',
})
.then(function (session) {
db = session.getSchema('test');
// Create a new collection 'my_collection'
return db.createCollection('my_collection');
})
.then(function (myColl) {
// Insert documents
return Promise
.all([
myColl.add({ name: 'Sakila', age: 15 }).execute(),
myColl.add({ name: 'Susanne', age: 24 }).execute(),
myColl.add({ name: 'User', age: 39 }).execute()
])
.then(function () {
// Find a document
return myColl
.find('name like :name && age < :age')
.bind({ name: 'S%', age: 20 })
.limit(1)
.execute(function (doc) {
// Print document
console.log(doc);
});
});
})
.then(function(docs) {
// Drop the collection
return db.dropCollection('my_collection');
})
.catch(function(err) {
// Handle error
});

28. C++ 28
// Connect to server
var mySession = MySQLX.GetSession("server=localhost;port=33060;user=user;password=password;");
var myDb = mySession.GetSchema("test");
// Create a new collection "my_collection"
var myColl = myDb.CreateCollection("my_collection");
// Insert documents
myColl.Add(new { name = "Sakila", age = 15}).Execute();
myColl.Add(new { name = "Susanne", age = 24}).Execute();
myColl.Add(new { name = "User", age = 39}).Execute();
// Find a document
var docs = myColl.Find("name like :param1 AND age < :param2").Limit(1)
.Bind("param1", "S%").Bind("param2", 20).Execute();
// Print document
Console.WriteLine(docs.FetchOne());
// Drop the collection
myDb.DropCollection("my_collection");

29. Java 29
// Connect to server
Session mySession = new
SessionFactory().getSession("mysqlx://localhost:33060/test?user=user&password=password");
Schema myDb = mySession.getSchema("test");
// Create a new collection 'my_collection'
Collection myColl = myDb.createCollection("my_collection");
// Insert documents
myColl.add("{"name":"Sakila", "age":15}").execute();
myColl.add("{"name":"Susanne", "age":24}").execute();
myColl.add("{"name":"User", "age":39}").execute();
// Find a document
DocResult docs = myColl.find("name like :name AND age < :age")
.bind("name", "S%").bind("age", 20).execute();
// Print document
DbDoc doc = docs.fetchOne();
System.out.println(doc);
// Drop the collection
myDB.dropCollection("test", "my_collection");

30. 30
New Shell

31. Starting using MySQL in few minutes 31

32. Quickly add a document 32

33. Find that document 33

34. Fast modifications 34

35. Shell info 35

36. For this example, I will use the well known restaurants collection:
We need to dump the data to a file and
we will use the MySQL Shell
with the Python interpreter to load the data.
Migration from MongoDB to MySQL Document Store
36

37. Dump and load using MySQL Shell & Python
This example is inspired by @datacharmer's work: https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo quiet eval 'DBQuery.shellBatchSize=30000;
db.restaurants.find().shellPrint()'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
37

38. Or use new bulk loader in 8.0.13
38

39. BSON Support
Now, it supports the conversion of the following additional BSON types:
■ Date
■ Timestamp
■ NumberDecimal
■ NumberLong
■ NumberInt
■ Regular Expression
■ Binary
39
> util.importJson("/path_to_file/neighborhoods_mongo.json",
{schema: "test", collection: "neighborhoods",
convertBsonTypes: true});

40. 40

41. 41
Let’s query
Too many records to show here … let’s limit it!
restaurants.find().limit(1)

42. 42
More Examples!
restaurants.find().fields([“name”,”cuisine”]).limit(2
)

43. 43
Comparing Syntax: MongoDB vs MYSQL
MongoDB:
> db.restaurants.find({"cuisine": "French",
"borough": { $not: /^Manhattan/} },
{"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2)
MySQL:
>restaurants.find(“cuisine=’French’ AND
borough!=’Manhattan’”).fields([“name”,”cuisine”,”borough”
]).limit(2)

44. 44
CRUD Operations

45. 45
Add a Document

46. 46
Modify a Document

47. 47
Remove a Document

48. 48
Find a Document

49. 49
MySQL Document Store Objects Summary

50. MySQL Document Store is Fully ACID Compliant 50

51. MySQL Document Store is Fully ACID Compliant 51

52. What about old SQL? The Hidden Part of the Iceberg 52

53. ★ Native data type (since 5.7.8)
★ JSON values are stored in MySQL tables using UTF8MB4
★ Conversion from "native" SQL types to JSON values
★ JSON manipulation functions (JSON_EXTRACT, JSON_KEYS,
JSON_SEARCH, JSON_TABLES, ...)
★ Generated/virtual columns
○ Indexing JSON data
○ Foreign Keys to JSON data
○ SQL Views to JSON data
JSON datatype is behind the scene
53

54. How Does It Work?? 54

55. What does a collection look like on the server ? 55

56. Every document has a unique identifier called the document ID, which can be
thought of as the equivalent of a table's primary key. The document ID value can
be manually assigned when adding a document.
If no value is assigned, a document ID is generated and assigned to the
document automatically !
Use getDocumentId() or getDocumentIds() to get _ids(s)
_id
56

57. Mapping to SQL Examples
createCollection('mycollection')
versus
CREATE TABLE `test`.`mycoll` (
doc JSON,
_id VARCHAR(32)
GENERATED ALWAYS AS (doc->>'$._id')
STORED
PRIMARY KEY
) CHARSET utf8mb4;
57

58. Mapping to SQL Examples
mycollection.add({‘test’: 1234})
versus
INSERT INTO `test`.`mycoll` (doc)
VALUES ( JSON_OBJECT( 'test',1234));
58

59. More Mapping to SQL Examples
mycollection.find("test > 100")
Versus
SELECT doc
FROM `test`.`mycoll`
WHERE (JSON_EXTRACT(doc,'$.test')
>100);
59

60. 60
SQL and JSON Example

61. It's also possible to create indexes without using SQL syntax 61

62. SQL and JSON Example (2): validation 62

63. SQL and JSON Example (3): explain 63

64. SQL and JSON Example (3): explain 64

65. SQL and JSON Example (4): add index 65

66. SQL and JSON Example (4): add index 66

67. [
{
"date": {
"$date": 1416009600000
},
"grade": "Z",
"score": 38
},
{
"date": {
"$date": 1398988800000
},
"grade": "A",
"score": 10
},
{
"date": {
"$date": 1362182400000
},
"grade": "A",
"score": 7
},
{
"date": {
"$date": 1328832000000
},
"grade": "A",
"score": 13
}
] 67
Embedded
Arrays of
values can be
messy to
traverse.

68. SQL and JSON Example (5): arrays 68

69. $.grades[0]
$.greades[1 to 2]
$.grades[first]
$.grades[last]
$.grades[first to last - 1]
69
Arrays are now simple

70. NoSQL as SQL 70
JSON_TABLE turns your un-
structured JSON data into a
temporary structured table!

71. NoSQL as SQL 71
This temporary structured table can
be treated like any other table --
LIMIT, WHERE, GROUP BY ...

72. 72
More Sophisticated Analysis

73. Find the top 10 restaurants by grade for each cuisine 73
WITH cte1 AS
(SELECT doc->>"$.name" AS name,
doc->>"$.cuisine" AS cuisine,
(SELECT AVG(score) FROM
JSON_TABLE(doc, "$.grades[*]" COLUMNS
(score INT PATH "$.score")) AS r) AS avg_score
FROM restaurants)
SELECT *, RANK() OVER
(PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank`
FROM cte1
ORDER BY `rank`, avg_score DESC LIMIT 10;
This query uses a Common Table Expression (CTE) and a Windowing Function to rank the
average scores of each restaurant, by each cuisine assembled in a JSON_TABLE

74. Conclusion: What Do I Gain?
74

75. This is the best of the two worlds in one product !
● Data integrity
● ACID Compliant
● Transactions
● SQL
● Schemaless
● flexible data structure
● easy to start (CRUD)
75

76. Mutable Data!!
Reduce Many to many joins
Replace ‘stub’ tables
Change on the fly, aggregate new data
76

77. Non JSON Data Transforms to JSON
77

78. And JSON to Relational
SELECT country_name, IndyYear FROM countryinfo,
JSON_TABLE(doc,"$" COLUMNS (country_name CHAR(60) PATH
"$.Name",IndyYear INT PATH "$.IndepYear")) AS stuff
WHERE IndyYear > 1992;
+----------------+----------+
| country_name | IndyYear |
+----------------+----------+
| Czech Republic | 1993 |
| Eritrea | 1993 |
| Palau | 1994 |
| Slovakia | 1993 | 78

79. GeoJSON support too!
mysql> SELECT ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111
12.22222)'),2);
+-------------------------------------------------------------+
| ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)'),2) |
+-------------------------------------------------------------+
| {"type": "Point", "coordinates": [11.11, 12.22]} |
+-------------------------------------------------------------+
79

80. New in MySQL 8.0
1. True Data Dictionary
2. Default UTF8MB4
3. Windowing Functions, CTEs, Lateral Derived Joins
4. InnoDB SKIPPED LOCK and NOWAIT
5. Instant Add Column
6. Histograms
7. Resource Groups
8. Better optimizer with new temporary table engine
9. True Descending Indexes
10.3D GIS
11.JSON Enhancements
80

81. Please buy my book!
If you deal with the JSON
Data Type or have an
interest in the MySQL
Document Store, this text is a
great guide with many
examples to help you
understand the complexities
and opportunities with a
native JSON Data Type
81

82. Thanks!
Contact info:
Dave Stokes
David.Stokes@Oracle.com
@Stoker
slideshare.net/davidmstokes
Elepantdolphin.blogger.com
opensourcedba.Wordpress.com
82