Más contenido relacionado La actualidad más candente (20) Similar a MySQL Document Store for Modern Applications (20) Más de Olivier DASINI (12) MySQL Document Store for Modern Applications1. MySQL Document Store
for Modern Applications
A Document Store with all the benefits of a Transactional RDBMS
© 2021 Oracle MySQL
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blogs : http://dasini.net/blog/en
: http://dasini.net/blog/fr
2. Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
2 © 2021 Oracle MySQL
3. 3
3
Olivier DASINI
Copyright © 2021 Oracle and/or its affiliates. All rights reserved.
MySQL Geek
Addicted to MySQL for 15+ years
Playing with databases for 20+ years
MySQL Writer, Blogger and Speaker
Also former : DBA, Consultant, Architect, Trainer, ...
MySQL Principal Solutions Architect EMEA at Oracle
Stay up to date!
Twitter: @freshdaz
Blog: www.dasini.net/blog/en
Me, Myself & I
4. Why MySQL ?
Program agenda
1
2
3
RDBMS & NoSQL concepts
MySQL Document Store
4 © 2021 Oracle MySQL
Demo
MySQL Enterprise Edition
4
5
5. 5 © 2021 Oracle MySQL
The world’s most popular open source database
6. Some Of Our Customers
MySQL powers the most demanding Web, E-commerce, SaaS and Online Transaction Processing
(OLTP) applications
It is a fully integrated transaction-safe, ACID compliant database with full commit, rollback, crash
recovery and row level locking capabilities
MySQL delivers the ease of use, scalability, and performance to power:
6 © 2021 Oracle MySQL
8. MySQL 8.0 Enables Modern Web Applications
9 © 2021 Oracle MySQL
Scalable & Stable
Better handling of high
contention, improved security,
and minimizing downtime
Data Driven
Optimizing services with real
time data analysis
Developer First
Hybrid data model and data
access APIs for flexibility for
developers
Mobile Friendly
Ready for location based
services. Handling Emoji
and Unicode characters
24x7
at Scale
10. Relational Databases
Data Integrity
– Normalization
– Constraints (Foreign keys, …)
Atomicity, Consistency, Isolation, Durability
– ACID Compliant
– Transactions
SQL
– Powerful Query Language
– Schema, Table, Row
– SELECT / UPDATE / DELETE / INSERT
11 © 2021 Oracle MySQL
Terminologies
11. J
JavaS
Script O
Object N
Notation
JSON is a lightweight data-interchange format
It is easy for humans to read and write
It is easy for machines to parse and generate
It is based on a subset of the JavaScript Programming
Language, Standard ECMA-262 3rd Edition – Dec. 1999
Use for storing and exchanging data
12 © 2021 Oracle MySQL
What is a JSON Document ?
An object that can represent structured data
Structure is implicit in the document; usually no external/central schema
Compact, popular and standardized
Can be represented natively in many languages (JavaScript,Python, etc.)
12. MySQL JSON Support
Native File Format
– Standardized as ECMA-404 (http://json.org)
– Binary Storage
Generated Columns
30+ Functions
– Search Functions
– Aggregations Functions
Query structured data and semi-structured JSON data
13 © 2021 Oracle MySQL
13. MySQL JSON Functions
14 © 2021 Oracle MySQL
Perform operations on JSON documents
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE_PATCH()
JSON_MERGE_PRESERVE()
JSON_OBJECT()
JSON_OVERLAPS()
JSON_PRETTY()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SCHEMA_VALID()
JSON_SCHEMA_VALIDATION_REPORT()
JSON_SEARCH()
JSON_SET()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_TABLE()
JSON_TYPE()
JSON_UNQUOTE()
JSON_MERGE_PATCH()
JSON_VALID()
JSON_VALUE()
MEMBER OF()
JSON_ARRAYAGG()
JSON_OBJECTAGG()
https://dev.mysql.com/doc/refman/8.0/en/json-functions.html
14. NoSQL : Document Store
Schemaless
– No schema design, No normalization, No Foreign Keys, No data types, …
– Very quick initial development
Flexible data structure
– Embedded arrays or objects
– Valid solution when natural data can’t be modeled optimally into a relational model
– Objects persistence without the use of any ORM – Mapping Object-Oriented
JSON
– Close to frontend
– Native in JavaScript
– Easy to learn
15 © 2021 Oracle MySQL
15. How DBAs see data
16 © 2021 Oracle MySQL
How Devs see data
{
" G N P " : 2 7 3 1 ,
" _ i d " : " M T Q " ,
" N a m e " : " M a r t i n i q u e " ,
" I n d e p Ye a r " : n u l l ,
" g e o g r a p h y " : {
" R e g i o n " : " C a r i b b e a n " ,
" C o n t i n e n t " : " N o r t h A m e r i c a " ,
" S u r f a c e A r e a " : 11 2 8
} ,
" g o v e r n m e n t " : {
" H e a d O f S t a t e " : " E m m a n u e l M a c r o n " ,
" G o v e r n m e n t F o r m " : " O v e r s e a s D e p a r t m e n t o f F r a n c e "
} ,
" d e m o g r a p h i c s " : {
" P o p u l a t i o n " : 3 9 5 0 0 0 ,
" L i f e E x p e c t a n c y " : 7 8 . 3 0 0 0 0 3 0 5 1 7 5 7 8 1
}
}
16. Document Oriented Databases
Schema (Database)
Collection
– Group of JSON documents.
– Equivalent of table
Document
– Set of key-value pairs in JSON format
– Equivalent of row (tuple)
© 2021 Oracle MySQL
Terminologies
CRUD (basic functions of persistent storage)
– CREATE
– READ
– UPDATE
– DELETE
17
17. Document Oriented Databases
Schemaless: No centralized database schema
– Data model enforcement and validation (if any) at application layer
– Simpler schema updates (no ALTER TABLE penalty)
NoSQL APIs: Simpler programming interfaces
– No specialized language for queries and data manipulation
– Complex queries handled at application layer (no complex SELECTs, JOINs)
– Document in, document out, manipulations at client side
Scalability, but some drawbacks:
– Limited database features (no foreign keys, no transactions, etc.)
– Weak consistency guarantees
© 2021 Oracle MySQL
Usability & Scalability
18
20. Using MySQL Document Store
Combining Relational & Document Stores
https://www2.slideshare.net/lefred.descamps/mysql-80-document-store-discovery-of-a-new-world
21. MySQL Document Store
22 © 2021 Oracle MySQL
An easy, straightforward way to
work with JSON documents in MySQL
www.dasini.net/blog/2019/04/02/mysql-json-document-store/
22. NoSQL, MySQL, why not...
Have both schema-less AND schema in the same technology stack?
One that checks all the boxes of all stakeholders:
24 © 2021 Oracle MySQL
Developers:
[x] Schemaless or/and Schema
[x] Rapid Prototyping/Simpler APIs
[x] Document Model
[x] Transactions
Operations:
[x] Performance Management/Visibility
[x] Robust Replication, Backup, Restore
[x] Comprehensive Tooling Ecosystem
[x] Simpler application schema upgrades
Business Owner:
[x] Don’t lose my data = ACID transactions
[x] Capture all my data = Extensible/Schemaless
[x] Products On Schedule/Time to Market = Rapid Development
NoSQL + SQL = MySQL
23. MySQL Document Store: Components
MySQL X Plugin
Introduces X Protocol for relational & document
operations
Maps CRUD operations to standard SQL
(relational tables, JSON datatype and functions)
X Protocol
New MySQL client protocol based on top of
industry standard (Protobuf)
Works for both, CRUD and SQL operations
MySQL InnoDB Cluster
A complete High Availability solution for MySQL
X DevAPI
New, modern, async developer API for CRUD &
SQL operations on top of X Protocol
Introduces collections as new schema object
MySQL Shell
Offers interactive X DevAPI mode for app
prototyping
MySQL Connectors
Support for X DevAPI for
● JavaScript, Python, PHP, Java, C#, C++
25 © 2021 Oracle MySQL
NoSQL + SQL = MySQL
25. MySQL Covers 4 Core Areas
28 © 2021 Oracle MySQL
Relational
MySQL Tables and SQL
Relational
MySQL Tables and SQL
Relational Analytics
MySQL Tables and SQL
Relational Analytics
MySQL Tables and SQL
Document Store
MySQL Collections and
Table with JSON Datatype
Key/Value
MySQL Memcached
Document Store
MySQL Collections and
Table with JSON Datatype
Key/Value
MySQL Memcached
Mass Scale Analytics
Relational and/or
Document Store
Mass Scale Analytics
Relational and/or
Document Store
NoSQL
Operational
SQL / Relational
Analytical
Combining Relational & Document Stores Workloads
26. Welcome to the X DevAPI!
Motivation
●
We are doing something that has not been done before
●
Document databases exist! Relational databases exist!
●
We even see databases that support relational and document querying over the same data set
●
However we have yet to see a relational database include a document model so that a user can use
document objects alongside their existing relational data
30 © 2021 Oracle MySQL
MySQL X devAPI connector for:
● Java : https://insidemysql.com/connector-j-8-0-11-the-face-for-your-brand-new-document-oriented-database/
● .NET : https://insidemysql.com/introducing-connector-net-with-full-support-for-mysql-8-0/
● Node.JS : https://insidemysql.com/introducing-connector-node-js-for-mysql-8-0/
● C++ : https://insidemysql.com/what-is-new-in-connector-c-8-0/
● Python : https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
● PHP : https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
● ODBC : https://insidemysql.com/what-is-new-in-connector-odbc-8-0/
SQL is now optional!
27. Welcome to the X DevAPI! - Python Example
31 © 2021 Oracle MySQL
Writing a program using MySQL 8.0 Document Store
import mysqlx
session = mysqlx.get_session({
"host": "localhost",
"port": 33060,
"user": "my_user",
"password": "s0S3kR*T"
})
schema = session.get_schema("test")
collection = schema.get_collection("my_collection")
result = collection.find("name like :param").bind("param", "Olivier").limit(1).execute()
docs = result.fetch_all()
print("Name: {0}".format(docs[0]["name"]))
session.close()
Connector/Python 8.0 installation
shell> pip install mysql-connector-python
https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
SQL is now optional!
28. Welcome to the X DevAPI! - PHP Example
32 © 2021 Oracle MySQL
Writing a program using MySQL 8.0 Document Store
$user = 'my_user';
$passwd = 's0S3kR*T';
$host = 'localhost';
$port = '33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
$session = mysql_xdevapigetSession($connection_uri);
$schema = $session->getSchema("test");
$collection = $schema->getCollection("my_collection");
$result = $collection->find("Name like :param")->bind(["param" => "Olivier"])->execute();
$docs = $result->fetchAll();
//… print results …
$session->close();
Requirements / Installation
https://dev.mysql.com/doc/apis-php/en/apis-php-mysql-xdevapi.setup.html
https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
SQL is now optional!
29. Using MySQL Document
Store with MySQL Shell
33 © 2021 Oracle MySQL
An integrated development & administration shell
where all MySQL products will be available through
a common scripting interface
30. MySQL Shell
●
DevOps Tool
– Designed for DevOps operations
●
Scripting for Javascript, Python, and SQL mode
– Rapid prototyping capabilities
●
InnoDB Cluster Support
– Setup your HA solution within minutes
●
Supports MySQL Standard and X Protocols
●
Document and Relational Models
– CRUD Document and Relational APIs via scripting
●
Traditional Table, JSON, Tab Separated output results formats
●
Both Interactive and Batch operations
34 © 2021 Oracle MySQL
Interface for Development and Administration of MySQL
31. MySQL Shell Overview
35 © 2021 Oracle MySQL
MySQL
Server 5.7
MySQL 8.0
Upgrade Checker
Prompt Themes
Auto Completion
&
Command History
MySQL
Server 8.0
Document
Store
X Dev API
SQL CLI
Output Formats
(Table, JSON,
Tabbed)
Batch Execution
JavaScript
Python
SQL
importJSON
ParalleI
importTABLE
InnoDB Cluster/
ReplicaSet
etc...
32. 36 © 2021 Oracle MySQL
js> session.createSchema('demo')
js> use demo
js> db.getCollections()
js> db.createCollection('myCollection')
js> db.getCollections()
js> db.myCollection.add({"param1":"value1", "param2":"value2"})
js> db.myCollection.find()
js> db.myCollection.find().limit(1)
js> db.myCollection.find("_id = '00005af018430000000000000002'")
js> db.myCollection.modify("_id = '1234'").set("param","value")
js> db.myCollection.remove("_id = '1234'")
js> session.startTransaction()
js> …
js> session.rollback()
js> db.myCollection.createIndex("ageIdx",{fields:[{"field":"$.age","type":"INT",required:false}]})
Copyright @ 2019 Oracle and/or its affiliates. All rights reserved.
MySQL Document Store “MySQL Shell’s cheat sheet”
C
Create
R
Read
U
Update
D
Delete
Index
T
Transaction
35. MySQL JSON datatype & Document Store API
39 © 2021 Oracle MySQL
Hybrid API
SQL Function
Data Type
MySQL X DevAPI
JSON Functions
JSON Datatype
Hybrid CRUD API of both
SQL and NoSQL provides
more flexibility for
development
Various SQL functions
to search and modify
JSON. Analysing JSON with
SQL by converting into
table with JSON_TABLE()
Seamlessly managing
“unstructured” data in
RDBMS tables with
efficient update
performance
{ } ();
36. ✔ Built on Proven SQL/InnoDB/Replication technology
✔ Schemaless / Relational / Hybrid
✔ Data Integrity – ACID Compliant – Transactions
✔ No SQL required – CRUD/JSON/Documents/Indexes
✔ Modern Dev API – Easy to start
✔ Modern/Efficient Protocol
✔ SQL Queries/Analytics over JSON Documents
✔ Transparent and Easy HA with MySQL InnoDB Cluster
MySQL Document Store
NoSQL + SQL = MySQL
40 © 2021 Oracle MySQL
37. MySQL Covers 4 Core Areas
41 © 2021 Oracle MySQL
Relational
MySQL Tables and SQL
Relational
MySQL Tables and SQL
Relational Analytics
MySQL Tables and SQL
Relational Analytics
MySQL Tables and SQL
Document Store
MySQL Collections and
Table with JSON Datatype
Key/Value
MySQL Memcached
Document Store
MySQL Collections and
Table with JSON Datatype
Key/Value
MySQL Memcached
Mass Scale Analytics
Relational and/or
Document Store
Mass Scale Analytics
Relational and/or
Document Store
NoSQL
Operational
SQL / Relational
Analytical
This is the best of the two worlds in one product !
38. MySQL Document Store
42 © 2021 Oracle MySQL
Works transparently with MySQL Enterprise Features
Clients and Applications
InnoDB Storage Engine
mysqld process
Firewall
Audit
Encryption
Authentication
Online Backup
Monitoring
Integration
Support
3rd
Party
Tools
3rd
Party
Tools
3rd
Party
Tools
3rd
Party
Tools
3rd
Party
Tools
NoSQL
NoSQL
Simple access
patterns
Compromise on consistency
for performance
Ad-hoc data format
Simple operation
SQL
SQL
Complex queries with
joins
ACID transactions
Well defined schemas
Rich set of tools
Masking
40. MySQL Enterprise Edition
44 © 2021 Oracle MySQL
Management Tools
Advanced Features Support
• Authentication
• Audit
• Encryption / TDE
• Firewall
• Data Masking
• High Availability
• Monitoring
• Backup
• Development
• Administration
• Migration
• Technical Support
• Consultative Support
• Oracle Certifications
Highest Levels of MySQL Security, Manageability and Reliability
41. MySQL Enterprise Edition
MySQL Enterprise Authentication
– External Authentication Modules
●
Microsoft AD, Linux PAMs, Native LDAP
MySQL Enterprise Encryption
– Public/Private Key Cryptography
– Asymmetric Encryption
– Digital Signatures, Data Validation
MySQL Enterprise Firewall
– Block SQL Injection Attacks
– Intrusion Detection
MySQL Enterprise Audit
– User Activity Auditing, Regulatory Compliance
MySQL Enterprise Monitor
– Changes in Database Configurations, Users
Permissions, Database Schema, Passwords
MySQL Enterprise Backup
– Securing Backups, AES 256 encryption
MySQL Enterprise TDE
– AES 256 encryption
– Key Management
MySQL Enterprise De-Identification
– Data Masking and Obfuscation
– Formatted Data Randomization
– Pseudonymization, Data Blacklists
– Random Data generation
45 © 2021 Oracle MySQL
More information available at :
Designed to provide monitoring, online backup, as well as improved security and scalability
43. Resources – MySQL Blogs
47 © 2021 Oracle MySQL
• Top 10 reasons for NoSQL with MySQL
– http://lefred.be/content/top-10-reasons-for-nosql-with-mysql/
• MySQL Document Store
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://github.com/mysql/mysql-connector-nodejs
– https://www.npmjs.com/package/@mysql/xdevapi
– https://dev.mysql.com/doc/refman/8.0/en/document-store.html
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://dev.mysql.com/doc/dev/connector-nodejs/8.0/
– https://www.mysql.com/news-and-events/web-seminars/mysql-document-store-and-node-js/
• MySQL Shell User Guide
– https://dev.mysql.com/doc/mysql-shell-excerpt/8.0/en/index.html
– https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/
• MySQL Shell Documentation
– https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/
• X Dev API User Guide
– https://dev.mysql.com/doc/x-devapi-userguide/en/
• X Plugin
– https://dev.mysql.com/doc/refman/8.0/en/x-plugin.html
• MySQL JSON Data Type
– https://dev.mysql.com/doc/refman/8.0/en/json.html
44. Resources – MySQL Blogs
48 © 2021 Oracle MySQL
• Common Table Expressions (CTE)
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-two-how-to-generate-series/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-three-hierarchies/
– http://mysqlserverteam.com/mysql-8-0-1-recursive-common-table-expressions-in-mysql-ctes-part-four-depth-first-or-breadth-first-traversal-
transitive-closure-cycle-avoidance/
• Window Functions
– http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
– https://mysqlserverteam.com/row-numbering-ranking-how-to-use-less-user-variables-in-mysql-queries/
– https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777
• Security
– https://www.mysql.com/news-and-events/web-seminars/checking-and-hardening-mysql-8-0-security-via-sql-and-without-os-access/
– https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/
– http://lefred.be/content/mysql-8-0-listing-roles/
– https://dev.mysql.com/doc/refman/8.0/en/password-management.html
• Upgrade from 5.7 to 8.0
– https://mysqlserverteam.com/inplace-upgrade-from-mysql-5-7-to-mysql-8-0/
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/
45. Resources – MySQL Blogs
49 © 2021 Oracle MySQL
• The complete list of new features in MySQL 8.0
– https://mysqlserverteam.com/the-complete-list-of-new-features-in-mysql-8-0/
• MySQL Server Team’s blog
– http://mysqlserverteam.com/
• MySQL High Availability Team’s blog
– http://mysqlhighavailability.com/
• MySQL Release Engineering Blog
– https://mysqlrelease.com/
• MySQL Dev Team Blog
– https://insidemysql.com/
• MySQL Enterprise Team Blogs
– https://blogs.oracle.com/mysql/
• MySQL Performance by Dimitri
– http://dimitrik.free.fr/blog/
• Lefred’s blog (MySQL Community Manager)
– http://lefred.be/
• Olivier Dasini’s blog
– http://dasini.net/blog/
46. 50 © 2021 Oracle MySQL
Questions?
olivier.dasini@oracle.com
47. Thank you
51 © 2021 Oracle MySQL
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blog : http://dasini.net/blog/en