Más contenido relacionado La actualidad más candente (20) Similar a Postgres NoSQL - Delivering Apps Faster (20) Postgres NoSQL - Delivering Apps Faster1. © 2014 EnterpriseDB Corporation. All rights reserved. 1
Postgres NoSQL - Delivering
Applications Faster
Marc Linster – SVP, Products and Services
2. © 2014 EnterpriseDB Corporation. All rights reserved. 2
• EDB Introduction
• BigData and NoSQL:
Volume, Variety and
Velocity
• Postgres – NoSQL for the Enterprise
• Performance Evaluation – Phase 2
• FDW – Coexistence of NoSQL only and SQL
• Leveraging a proven skill set to create Web 2.0
applications
• Postgres as a NoSQL/SQL Integration Platform
Agenda
3. © 2014 EnterpriseDB Corporation. All rights reserved. 3
POSTGRES
innovation
ENTERPRISE
reliability
24/7
support
Services
& training
Enterprise-class
features & tools
Indemnification
Product
road-map
Control
Thousands
of developers
Fast
development
cycles
Low cost
No vendor
lock-in
Advanced
features
Enabling commercial
adoption of Postgres
4. © 2014 EnterpriseDB Corporation. All rights reserved. 4
Postgres Plus
Advanced Server Postgres Plus
Cloud Database
High Availability
PerformanceManagement
REMOTE
DBA 24x7
SUPPORT
PROFESSIONAL
SERVICES
TRAINING
EDB Serves
All Your Postgres Needs
PostgreSQL
Security
5. © 2014 EnterpriseDB Corporation. All rights reserved. 5
• Common Myth (or misconception)
− Volume, Velocity, Variety and ACID are not compatible!
− You cannot have it all!
• Their (incorrect) reasoning:
− Very High Data Volumes (log analysis, append only applications,
social media) applications cannot be handled by relational systems
− The data is sent to the application at high speeds and has to be
ingested quickly – not in batch mode
− Web 2.0 applications require more than SQL – they also need
documents, IMs, emails, graphs, links
− ACID (Atomicity, Consistency, Isolation and Durability) compliant
systems are too inflexible, too slow and cannot handle the new
data types
BigData, NoSQL and SQL: Myths about
Volume, Variety, Velocity and ACID
6. © 2014 EnterpriseDB Corporation. All rights reserved. 6
$$$$$$
ACID, Volume, Velocity and Variety in Context
ACID Transactional
Ingestion Velocity
Variety
Data Volume
Postgres
Data Mining/BigData
Document/NoSQL-
Only
7. © 2014 EnterpriseDB Corporation. All rights reserved. 7
• Performance Benchmarks
− Postgres is very fast and can handle huge
amounts of data
− Postgres can selectively relax key ACID features
to increase performance
• Data Types
− Postgres has JSON, JSONB, Key-Value Pair,
plus arrays, ranges, timezones, dates, integer,
floating point, etc.
• Proven track record
− ACID compliant
− Open source
− ANSI SQL
− Huge developer and vendor community
Postgres –
NoSQL for the Enterprise
8. © 2014 EnterpriseDB Corporation. All rights reserved. 8
• HSTORE
− Key-value pair
− Simple, fast and easy
− Postgres v 8.2 – pre-dates many
NoSQL-only solutions
• JSON
− Hierarchical document model
− Introduced in Postgres 9.2, perfected in 9.3
• JSONB
− Binary version of JSON
− Faster, more operators and even more robust
− Postgres 9.4
NoSQL Data in Postgres
9. © 2014 EnterpriseDB Corporation. All rights reserved. 9
• Creating a table with a JSONB field
CREATE TABLE json_data (data JSONB);!
• Simple JSON data element:
{"name": "Apple Phone", "type": "phone", "brand":
"ACME", "price": 200, "available": true,
"warranty_years": 1}!
• Inserting this data element into the table json_data
INSERT INTO json_data (data) VALUES !
!(’ { !"name": "Apple Phone", !
! !"type": "phone", !
! !"brand": "ACME", !
! !"price": 200, !
! !"available": true, !
! !"warranty_years": 1 ! !!
!} ')!
JSON Examples
10. © 2014 EnterpriseDB Corporation. All rights reserved. 10
SELECT DISTINCT !
!data->>'name' as products !
FROM json_data;
!
products !
------------------------------!
Cable TV Basic Service Package!
AC3 Case Black!
Phone Service Basic Plan!
AC3 Phone!
AC3 Case Green!
Phone Service Family Plan!
AC3 Case Red!
AC7 Phone!
A simple query for JSON data
This query does not
return JSON data – it
returns text values
associated with the
key ‘name’
11. © 2014 EnterpriseDB Corporation. All rights reserved. 11
SELECT data FROM json_data;!
data !
------------------------------------------!
{"name": "Apple Phone", "type": "phone",
"brand": "ACME", "price": 200,
"available": true, "warranty_years": 1}!
A query that returns JSON data
This query returns the JSON data in its
original format
12. © 2014 EnterpriseDB Corporation. All rights reserved. 12
• JSON is naturally integrated with
ANSI SQL in Postgres
• JSON and HSTORE are elegant and easy to
use extensions of the underlying object-
relational model
• JSON and SQL queries use the same
language, the same planner, and the same
ACID compliant transaction framework
JSON and ANSI SQL –
A Great Fit
13. © 2014 EnterpriseDB Corporation. All rights reserved. 13
SELECT DISTINCT
product_type,
data->>'brand' as Brand,
data->>'available' as Availability
FROM json_data
JOIN products
ON (products.product_type=json_data.data->>'name')
WHERE json_data.data->>'available'=true;
product_type | brand | availability
---------------------------+-----------+--------------
AC3 Phone | ACME | true
JSON and ANSI SQL Example
ANSI SQL
JSON
No need for programmatic logic to combine SQL and
NoSQL in the application – Postgres does it all
14. © 2014 EnterpriseDB Corporation. All rights reserved. 14
Bridging between SQL and NoSQL
Simple ANSI SQL Table Definition
CREATE TABLE products (id integer, product_name text );!
Select query returning standard data set
SELECT * FROM products;
!
id | product_name !
----+--------------!
1 | iPhone!
2 | Samsung!
3 | Nokia!
Select query returning the same result as a JSON data set
SELECT ROW_TO_JSON(products) FROM products;
{"id":1,"product_name":"iPhone"}
{"id":2,"product_name":"Samsung"}
{"id":3,"product_name":"Nokia”}
15. © 2014 EnterpriseDB Corporation. All rights reserved. 15
• Start unstructured, and become
structured as you learn more
− Use the quick-to-get-started capabilities of NoSQL
− Complete the initial sprints without a DBA
− Move data between unstructured and structured
− Embrace corporate data standards as you move
from the stand-alone application towards integrated
applications with a bigger value proposition
Postgres Provides Great Flexibility
By 2017, 50% of data stored in NoSQL DBMSs will be damaging to
the business due to lack of applied information governance policies
and programs.
Gartner, December 2013
16. © 2014 EnterpriseDB Corporation. All rights reserved. 16
• Goal
− Help our customers understand when to chose Postgres and
when to chose a specialty solution
− Help us understand where the NoSQL limits of Postgres are
• Setup
− Compare Postgres 9.4 to Mongo 2.6
− Single instance setup on AWS M3.2XLARGE (32GB)
• Test Focus
− Data ingestion (bulk and individual)
− Data retrieval
Postgres NoSQL Performance Evaluation
Load Generator
Postgres 9.4
MongoDB 2.6
17. © 2014 EnterpriseDB Corporation. All rights reserved. 17
Postgres NoSQL Performance Evaluation
Generate JSON Documents
10 M documents & 50 M documents
Load into MongoDB 2.6
(IMPORT)
Load into
Postgres 9.4
(COPY)
10 Million individual
INSERT commands
10 Million individual
INSERT commands
Multiple SELECT
statements
Multiple SELECT
statements
T1
T2
T3
18. © 2014 EnterpriseDB Corporation. All rights reserved. 18
Test 1: 10 Million Documents
Test
Criterion
MongoDB
2.6
Postgres
9.4
Data
load
(s)
2,624
1,193
Inserts
(s)
16,491
5,637
Selects
(s)
187
67
DB
Size
(GB)
20.05
14.89
0%
50%
100%
150%
200%
250%
300%
350%
Data
load
(s)
Inserts
(s)
Selects
(s)
DB
Size
(GB)
Mongo
DB
2.4/Postgres
9.4
Rela9ve
Performance
Comparison
(10M
Documents)
MongoDB
2.6
Postgres
9.4
19. © 2014 EnterpriseDB Corporation. All rights reserved. 19
Test 2: 50 Million Documents
Test
Criterion
MongoDB
2.6
Postgres
9.4
Data
load
(s)
15,391
7,319
Inserts
(s)
85,639
29,125
Selects
(s)
1,929
753
DB
Size
(GB)
92.63
69.36
0%
50%
100%
150%
200%
250%
300%
350%
Data
load
(s)
Inserts
(s)
Selects
(s)
DB
Size
(GB)
Mongo
DB
2.4/Postgres
9.4
Rela9ve
Performance
Comparison
(50M
Documents)
MongoDB
2.6
Postgres
9.4
20. © 2014 EnterpriseDB Corporation. All rights reserved. 20
• Tests confirm that Postgres can handle many NoSQL
workloads
• EDB is making the test scripts publically available
• EDB encourages community participation to
better define where Postgres should be used
and where specialty solutions are appropriate
• Download the source at
https://github.com/EnterpriseDB/pg_nosql_benchmark
• Join us to discuss the findings at
http://bit.ly/EDB-NoSQL-Postgres-Benchmark
Postgres NoSQL Performance Evaluation
21. © 2014 EnterpriseDB Corporation. All rights reserved. 21
• FDW implements SQL/MED ("SQL
Management of External Data")
• PostgreSQL 9.1 - read-only support
• PostgreSQL 9.3 – read/write support
• FDW
− Makes data on other servers (or services) look like tables in
Postgres
− available for databases (MongoDB, MySQL, Oracle, …), files,
services (Twitter, …)
• MongoDB FDW: https://github.com/EnterpriseDB
Foreign Data Wrappers –
Co-Existence Platform
22. © 2014 EnterpriseDB Corporation. All rights reserved. 22
CREATE EXTENSION mongo_fdw;!
CREATE SERVER mongo_server
!FOREIGN DATA WRAPPER mongo_fdw
!OPTIONS (address '172.24.39.129', port '27017');!
CREATE USER MAPPING FOR enterprisedb
!SERVER mongo_server
!OPTIONS (username 'mongo', password 'mongo');!
CREATE FOREIGN TABLE mongo_data(
!name text,
!brand text,
!type text)
!SERVER mongo_server
!OPTIONS (
! !database 'benchmark',
! !collection 'json_tables');!
MongoDB FDW Example
23. © 2014 EnterpriseDB Corporation. All rights reserved. 23
SELECT * FROM mongo_data WHERE brand='ACME' limit 10;!
!
name | brand | type
-------------+-------+-------
AC7553 Phone | ACME | phone
AC7551 Phone | ACME | phone
AC7519 Phone | ACME | phone
AC7565 Phone | ACME | phone
AC7555 Phone | ACME | phone
AC7529 Phone | ACME | phone
AC7528 Phone | ACME | phone
AC7547 Phone | ACME | phone
AC7587 Phone | ACME | phone
AC7541 Phone | ACME | phone!
(10 rows)!
!
MongoDB FDW Example
24. © 2014 EnterpriseDB Corporation. All rights reserved. 24
INSERT INTO mongo_data(name, brand, type)
!VALUES('iphone6 phone','Apple Inc','phone');!
SELECT* FROM mongo_data WHERE brand='Apple Inc';!
_id | name | brand | type
--------------------------+----------------+-----------+-------
53ea4f59fe5586a15714881d | iphone6 phone | Apple Inc | phone!
!
UPDATE mongo_data SET brand='Apple Product'
!WHERE brand='Apple Inc’;!
SELECT * FROM mongo_data WHERE brand='Apple Product’;!
_id | name | brand | type
--------------------------+----------------+---------------+-------
53ea4f59fe5586a15714881d | iphone6 phone | Apple Product | phone!
!
MongoDB FDW Example
25. © 2014 EnterpriseDB Corporation. All rights reserved. 25
• Postgres allows you to
− Use the same Infrastructure DBAs (no change!!!)
− Use the same management, maintenance, backup, restore,
high availability, disaster recovery and compliance processes
for SQL and NoSQL
− Start unstructured and align with corporate governance
standards as your project matures
− Start unstructured (data logic in the application) and migrate
shared data logic into the database è enhances reuse
− Build on the existing PL/pgSQL programming skillset
− EDB Postgres Plus adds Oracle PL/SQL compatibility (fully
integrated with JSON and HSTORE)
Leveraging a Proven Skill Set
26. © 2014 EnterpriseDB Corporation. All rights reserved. 26
• Postgres has many NoSQL features without
the drawbacks:
− Schema-less data types, with sophisticated indexing
support
− Schema changes with rapid addition and removal of
keys and tags
− Durable by default, but configurable per-table or
per-transaction
− Foreign Data Wrappers (FDW) support co-existence
and tight integration
− Standards based with very low technology risk
− Highly available skill set
Postgres: The Best of Both Worlds
27. © 2014 EnterpriseDB Corporation. All rights reserved. 27
• Structures and standards emerge!
• Data has references (products link to catalogues;
products have bills of material; components appear in
multiple products; storage locations link to ISO country
tables)
• When the database has duplicate data entries, then the
application has to manage updates in multiple places –
what happens when there is no ACID transactional
model?
“No SQL Only” or “Not Only SQL”?
28. © 2014 EnterpriseDB Corporation. All rights reserved. 28
• Postgres is Not Only SQL (NoSQL is No SQL only)
• Fully ACID compliant
• Proven track record
• Fully capable of handling the variety, velocity and
volume requirements of most applications
• Tackle NoSQL projects without leaving the capabilities
of the relational model behind you
Postgres is the best of both worlds, and it’s open source!
Say ‘Yes’ to ‘Not Only SQL’
29. © 2014 EnterpriseDB Corporation. All rights reserved. 29
• Contact us for an Enterprise NoSQL Assessment
(sales@enterprisedb.com)
• Review EDB’s NoSQL Resources
− Whitepaper -
http://www.enterprisedb.com/wp-using-nosql-features-postgres
− NoSQL Solution Information -
http://www.enterprisedb.com/nosql-for-enterprise
− Benchmarks- https://github.com/EnterpriseDB