SlideShare una empresa de Scribd logo
1 de 71
SQL Strikes Back
©2016 Couchbase Inc. 2
©2016 Couchbase Inc. 3
Agenda
Thinking in N1QL
Anatomy of Query
©2016 Couchbase Inc. 4
©2016 Couchbase Inc. 5
Business Application Questions &Tasks
How many new customers we
got last month?
Get the list of stores in
customer region
Search stores for the shoe
customer is looking for?
Generate a list of shipment
due today
Load the new inventory data
Update the sale prices in
outlet stores only
Merge the customer lists
©2016 Couchbase Inc. 6
NoSQL Landscape
Document
• Couchbase
• MongoDB
• DynamoDB
• DocumentDB
Graph
• OrientDB
• Neo4J
• DEX
• GraphBase
Key-Value
• Couchbase
• Riak
• BerkeleyDB
• Redis
• … Wide Column
• Hbase
• Cassandra
• Hypertable
JSON to Represent Data
©2015 Couchbase Inc. 7
Data Management Landscape
Processing
in Files
MapReduce
Generic fileformats
Rows/Columns
in files (tables)
Hive – Pig - etc
Query
Impala
Hive
NoSQL
MongoDB
Couchbase
Hbase
Cassandra
HADOOP
(Analytical)
Disk &
Storage
Highly Structured Data
SQL, R, etc
Bytes & Blocks
$100K – $200K /TB$1K/TB$10K/TB
Semi Structured & Self describingNo Structure
OLTP EDW
$10K-$20K/TB
Drill
Operational Big data
Couchbase 4
©2016 Couchbase Inc. 8
Properties of Real-World Data
 Rich structure
– Attributes, Sub-structure
 Relationships
– To other data
 Value evolution
– Data is updated
 Structure evolution
– Data is reshaped
Customer
Name
DOB
Billing
Connections
Purchases
©2016 Couchbase Inc. 9
Transform: Relational to JSON
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
DocumentKey: CBL2015
CustomerID Name DOB
CBL2015 Jane Smith 1990-01-30
Customer
ID
Type Cardnum Expiry
CBL2015 visa 5827… 2019-03
CBL2015 maste
r
6274… 2018-12
CustomerID ConnId Name
CBL2015 XYZ987 Joe Smith
CBL2015 SKR007 Sam Smith
CustomerID item amt
CBL2015 mac 2823.52
CBL2015 ipad2 623.52
CustomerID ConnId Name
CBL2015 XYZ987 Joe Smith
CBL2015 SKR007 Sam
Smith
Contacts
Customer
Billing
ConnectionsPurchases
©2016 Couchbase Inc. 10
JSON
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
 JSON is a means to the end and not the end itself
– JSON is the representation of the enterprise data model for applications
– JSON flexibility translates to application flexibility
• Simple flattened data can be represented
• Entities with complex data, always accessed analyzed together should belong
together
– Applications are designed to handle the flexible data model.
©2016 Couchbase Inc. 11
Models for Representing Data
Data Concern Relational Model
JSON Document Model
(NoSQL)
Rich Structure
 Multiple flat tables
 Constant assembly / disassembly
 Documents
 No assembly required!
Relationships
 Represented
 Queried (SQL)
 Represented
 Queried? Not until now…
Value Evolution  Data can be updated  Data can be updated
Structure Evolution
 Uniform and rigid
 Manual change (disruptive)
 Flexible
 Dynamic change
©2016 Couchbase Inc. 12
ResultSet
©2016 Couchbase Inc. 13
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
LoyaltyInfo ResultDocuments
Orders
CUSTOMER
Built Manually; Expensive
©2016 Couchbase Inc. 14
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-2847-3909",
"expiry" : "2019-03"
},
{
"type" : "master",
"cardnum" : "6274-2842-2847-3909",
"expiry" : "2019-03"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
"Name" : "Joe Smith"
},
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
{
"CustId" : "PQR823",
"Name" : "Dylan Smith"
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
LoyaltyInfo ResultDocuments
Orders
CUSTOMER
©2016 Couchbase Inc. 15
N1QL Changes the Possibilities for NoSQL
©2016 Couchbase Inc. 16
Goal of N1QL: SQL for JSON
Give developers and enterprises an
expressive, powerful, and complete
language for querying, transforming,
and manipulating JSON data.
©2016 Couchbase Inc. 17
N1QL: Developers & Enterprises
Application Developers in all languages
–Couchbase SDK Support for N1QL
–Open REST API
Exchanges data with other databases using Standard
Tools
Simba provides ODBC, JDBC drivers
Tableau, INFA interoperability
©2016 Couchbase Inc. 18
N1QL: expressive
 Access to every part of JSON document
 Scalar & Aggreate functions
 Issue subquery in any expressions
 Subqueries
 Subqueries in the FROM clause
©2016 Couchbase Inc. 19
N1QL: powerful
 Access to every part of JSON document
 JOINS, Aggregations, standard scalar functions
 Aggregation on arrays
 NEST & UNNEST operations
 Covering Index
©2016 Couchbase Inc. 20
N1QL: querying
 INSERT
 UPDATE
 DELETE
 MERGE
 SELECT
 EXPLAIN
©2016 Couchbase Inc. 21
N1QL: transforming & manipulating
 FullTransformation of the data via Query.
 INSERT
– INSERT single & multiple documents
– INSERT result a SELECT statement
 DELETE documents based on complex filter
 UPDATE any part of JSON document & use complex filter.
 MERGE two sets of documents using traditional MERGE
statement
©2016 Couchbase Inc. 22
©2016 Couchbase Inc. 23
Where’s My Beer?
SELECT {"Mybeer": "My Beer is " || beer.name || "."},
ARRAY_AGG({"name":brewery.name}) brewery,
ARRAY_AGG({"name":brewery.name, "state":brewery.state,
"city":brewery.city, "location":brewery.geo}) locations,
ARRAY_COUNT(ARRAY_AGG(brewery.name)) as brewery_count
FROM `beer-sample` beer
LEFT OUTER JOIN `beer-sample` brewery
ON KEYS beer.brewery_id
WHERE beer.type = 'beer'
AND brewery.type = 'brewery'
AND brewery.state = 'California'
GROUP BY beer.name
ORDER BY ARRAY_COUNT(ARRAY_AGG(brewery.name)) desc,
beer.name asc
LIMIT 5
;
©2016 Couchbase Inc. 24
Hands-onWorkshop
 Execute the query1
 Observe & discuss the structure of the document returned by the query
 Run the EXPLAIN for the query
 Modify
– WHERE clause
– LIMIT clause
– ORDER BY clause
©2016 Couchbase Inc. 25
QUERY EXECUTION
©2016 Couchbase Inc. 26
N1QL: Query Execution Flow
Clients
1. Submit the query over RESTAPI 8. Query result
2. Parse, Analyze, create Plan 7. Evaluate: Documents to results
3. Scan Request;
index filters
6. Fetch the documents
Index
Servic
e
Query
Service
Data
Servic
e
4. Get qualified doc keys
5. Fetch Request,
doc keys
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
©2016 Couchbase Inc. 27
Inside a Query Service
Client
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
©2016 Couchbase Inc. 28
Inside a Query Service
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
Prepare
Phase
Projection
Phase
Run
Phase
Scan
Phase
Fetch
Phase
Sort
Phase
©2016 Couchbase Inc. 29
QUERY PREPARE
©2016 Couchbase Inc. 30
Prepare Phase: Parse & Semantic Check
 Analyzes the Query for syntax & grammar
 Only verifies for existence of referenced buckets
 Flexible schema means, you can refer to arbitrary attribute
names. N1QL key names are CaSe SENSitive.
 A key is NULL if it’s a known null. {“location”:null}
 A key is MISSING if not present in the document
 Use IS NULL and IS MISSING to check for each condition
 Full reference to JSON structure
– Nested reference: CUSTOMER.contact.address.state
– Array Reference: CUSTOMER.c_contact.phone_number[0]
 SQL is enhanced to access & manipulateArrays
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2016 Couchbase Inc. 31
Prepare: PLAN
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Each query can be executed in several ways
 Create the query execution plan
– Access path for each keyspace reference
– Decide on the filters to push down
– Determine Join order and join method
– Create the execution tree
 For each keyspace reference:
– Look at the available indices
– Match the filters in the query with index keys
– Choose one or more indices for each keyspace
©2016 Couchbase Inc. 32
PREPARE
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Ever statement executed in the engine is PREPARED
 When you have same statement executed millions of
times, you can save prepare time
 Prepare statement can use simple adhoc statement
PREPARE p1 FROM SELECT * from `beer-sample` LIMIT 1;
EXECUTE p1;
 Compare the execution time
 Prepare Statement can exploit parameters to bind
PREPARE p2 FROM SELECT * from `beer-sample` WHERE abv = $1;
$ curl http://localhost:8093/query/service -H "Content-
Type: application/json" -d '{"prepared":"p2",
"args":[5.0]}’
$ curl http://localhost:8093/query/service -d 'prepared
="mp"&args=[5.0]’
©2016 Couchbase Inc. 33
PREPARE
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Ever statement executed in the engine is PREPARED
 When you have same statement executed millions of
times, you can save prepare time
 Prepare statement can use simple adhoc statement
PREPARE p1 FROM SELECT * from `beer-sample` LIMIT 1;
EXECUTE p1;
 Compare the execution time
 Prepare Statement can exploit parameters to bind
PREPARE p2 FROM SELECT * from `beer-sample` WHERE abv = $1;
$ curl http://localhost:8093/query/service -H "Content-
Type: application/json" -d '{"prepared":"p2",
"args":[5.0]}’
$ curl http://localhost:8093/query/service -d 'prepared
="mp"&args=[5.0]’
©2016 Couchbase Inc. 34
PREPARE
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Parameter can be named or positional.
 Named parameters:
curl http://localhost:8093/query/service
-d 'statement=prepare p3 from select
count(*) from `beer-sample` where abv >
$val’
curl http://localhost:8093/query/service
-H "Content-Type: application/json" -d
'{"prepared":"p3", "$val":5.0}’
curl http://localhost:8093/query/service
-d 'prepared="p3"&$val=5.0'
©2016 Couchbase Inc. 35
PREPARE: Cluster Environment
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
curl http://localhost:8093/query/service
-d 'statement=prepare p4 from select name
from `beer-sample` where name LIKE $val’
curl http://localhost:8093/query/service -H "Content-Type:
application/json" -d '{"prepared":"p4",
"encoded_plan":"H4sIAAAJbogA/5xSTW/qMBD8K5F5B5B4T3oSp9x6KBJ
qD0gcq4q4yZAYHNtdOxSK0t/edaDlq5dys2dnZ2fW3gmY3BYo5k5LI1Ihhs
LIGnxyIz5bB5LBkkh3one8iBleG+4EUz7ySumCwN1P56S7JlSW1HtkdaSbV
KakaknbWc7+hkKZAhtGe24PM7TC1juZR88vAP31snYahyBflQIL2ejAaOOV
KRkpvRLt8GKWJKk19LehmxyPEfLqt8YurYyVDqC4OWsKFZSNr9PvZydK2b8
sKmWDRKsVkj9rqQdXOhPDzVJPyS6Rx/wEz/PmLF77zjo2LhJ/lhbt87Uzc9
TjcmQIfo610ijh48oOodLT2On/9lJpFgiy7kYIr0ojQ0OI/YcvuPScmmsBm
xB/JMFJQuJGyYJsnXhotpBE8h44S5C8VWByV32cPNx3+xHtZwAAAP//FkDT
W/ICAAA=", "$val":"%Ale%"}'
©2016 Couchbase Inc. 36
PREPARE: Summary
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Using prepare state saves time.
– Amount of saving depends on the query complexity, index
choices.
 Effective when the same statement or statement with
parameters are executed
 Encoded plan enables SDKs and applications to prepare
once and use it on any node in the cluster.
 SDKs use prepared statement behind the scenes.
©2016 Couchbase Inc. 37
Hands-onWorkshop
 Prepare the queries P1 through P4.
 Execute them in cbq and via curl
 Compare the execution timings.
©2016 Couchbase Inc. 38
QUERY PLANNING & INDEX SELECTION
©2016 Couchbase Inc. 39
PLAN:We’re not done yet!
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Each query can be executed in several ways
 Create the query execution plan
– Access path for each keyspace reference
– Decide on the filters to push down
– Determine Join order and join method
– Create the execution tree
 For each keyspace reference:
– Look at the available indices
– Match the filters in the query with index keys
– Choose one or more indices for each keyspace
©2016 Couchbase Inc. 40
Index Overview: Primary Index
 Primary Index
 CREATE PRIMARY INDEX `beer-sample`;
 Document key is unique for the bucket.
 Primary index is used when no other qualifying
index is available or when no predicate is given
in the query.
 PrimaryScan is equivalent of full table scan
"beer-sample": {
"abv": 7.2,
"brewery_id":
"21st_amendment_brewery_cafe",
"category": "North American Ale",
"description": "Deep golden color...!",
"ibu": 0,
"name": "21A IPA",
"srm": 0,
"style": "American-Style India Pale Ale",
"type": "beer",
"upc": 0,
"updated": "2010-07-22 20:00:20"
}
Document key: “guiness_d1”
©2016 Couchbase Inc. 41
"beer-sample": {
"abv": 7.2,
"brewery_id":
"21st_amendment_brewery_cafe",
"category": "North American Ale",
"description": "Deep golden color...!",
"ibu": 0,
"name": "21A IPA",
"srm": 0,
"style": "American-Style India Pale Ale",
"type": "beer",
"upc": 0,
"updated": "2010-07-22 20:00:20"
}
Document key: “Pale Ale.A1”
Index Overview: Secondary Index
Secondary Index can be created on any
combination of attribute names.
create index idxbeerabv on `beer-
sample`(abv, state);
create index idxtypeabvstate on
`beer-sample`(abv, state) where type =
'beer';
 Need to have matching indices with right key-
ordering
©2016 Couchbase Inc. 42
GSI Index: Key details
 SupportedTypes
 String, Boolean, Numeric, Nil,Array, Sub-document
 Comp
 Total length of the keys
 4 KB – actual length of the key indexed
 Number of keys
 4096!
©2016 Couchbase Inc. 43
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
create index idxbeerabv on
`beer-sample`(abv, state);
EXPLAIN SELECT count(*)
FROM `beer-sample`
WHERE abv > 5.0 AND state = 'California';
select *
from system:indexes
where keyspace_id = 'beer-sample';
 Explain provides the JSON representation of the
query plan
 Focus on the index selection and the predicates
pushed down
©2016 Couchbase Inc. 44
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
create index idxtypeabvstate on `beer-
sample`(abv, state) where type = ‘beer’;
select * from system:indexes where name =
'idxtypeabvstate';
"indexes": {
"condition": "(`type` = "beer")",
"datastore_id": "http://127.0.0.1:8091",
"id": "611ffca1720b7868",
"index_key": [
"`abv`",
"`state`"
],
"keyspace_id": "beer-sample",
"name": "idxtypeabvstate",
"namespace_id": "default",
"state": "online",
"using": "gsi"
}
©2016 Couchbase Inc. 45
Plan: Index selection – Partial Index
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
create index idxabvp1
on `beer-sample`(abv, state)
where type = 'beer' and abv > 5.0;
explain select count(*)
from `beer-sample`
where type = 'beer' and abv = 7.0 ;
©2015 Couchbase Inc. 46
Power Features: IntersectScan (Multi-Index Scan)
create index idxbeerabv on `beer-sample`(abv);
create index idxbeercat on `beer-sample`(category);
select *
from `beer-sample`
where abv > 15 and category like '%North%';
 IntersectScan using multiple indices:
– Multiple indices are scanned in parallel
– Provides more flexibility in using the indices for filters
– Requires less number of indexes defined on table.
• Can save on disk space and memory space as well.
©2016 Couchbase Inc. 47
IntersectScan method using multiple indices
create index idxbeerabv on `beer-sample`(abv);
create index idxbeercat on `beer-sample`(category);
select *
from `beer-sample`
where abv > 15 and category like '%North%';
 IntersectScan using multiple indices:
– Multiple indices are scanned in parallel
– Provides more flexibility in using the indices for filters
– Requires less number of indexes defined on table.
• Can save on disk space and memory space as well.
©2016 Couchbase Inc. 48
Plan Directives: USE INDEX
create index idxbeerabv on `beer-sample`(abv);
create index idxbeercat on `beer-sample`(category);
select *
from `beer-sample` USE INDEX (idxbeerabv)
where abv > 15 and category like '%North%';
select *
from `beer-sample` USE INDEX (idxbeerabv, idxbeercat)
where abv > 15 and category like '%North%’;
 USE INDEX suggests one or more appropriate indices
 Optimizer validates the indices. Primary Index is the backup index.
 Provides stability to the optimizer plan.
©2016 Couchbase Inc. 49
Query Execution: Fetch
 List of qualified document-keys are grouped into batches.
 List of the documents is obtained from the Index or specified
directly via USE KEYS clause.
 Fetch request is done in parallel.
 The join operation use the fetch operation to get the
matching document.
 Fetch results are streamed into next operators.
 For big queries, scan-fetch-join-filter-aggregation will be
executing in parallel.
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2016 Couchbase Inc. 50
JOINS
©2016 Couchbase Inc. 51
Query Execution: Join
 You can join any two key spaces if one has document-key of
the other.
 You can store multiple entities within the same bucket and
join between distinct groups
 Uses Nested Loop JOIN now
 JOINs are done in the same order specified in the query
 Index selection is important for the first keyspace in the
FROM clause.
 Qualified documents from that scan is joined with the other
Keyspace using the DOCUMENT KEYS
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 52
Query Execution: Join
"CUSTOMER": {
"C_D_ID": 10,
"C_ID": 1938,
"C_W_ID": 1,
"C_BALANCE": -10,
"C_CITY": ”San Jose",
"C_CREDIT": "GC”,
"C_DELIVERY_CNT": 0,
"C_DISCOUNT": 0.3866,
"C_FIRST": ”Jay",
"C_LAST": ”Smith",
"C_MIDDLE": "OE",
"C_PAYMENT_CNT": 1,
"C_PHONE": ”555-123-1234",
"C_SINCE": "2015-03-22 00:50:42.822518",
"C_STATE": ”CA",
"C_STREET_1": ”555, Tideway Drive",
"C_STREET_2": ”Alameda",
"C_YTD_PAYMENT": 10,
"C_ZIP": ”94501"
}
Document key: “1.10.1938” Document key: “1.10.143”
“ORDERS”: {
“O_CUSTOMER_KEY”: “1.10.1938):
"O_D_ID": 10,
"O_ID": 1,
"O_ALL_LOCAL": 1,
"O_CARRIER_ID": 2,
"O_C_ID": 1938,
"O_ENTRY_D": "2015-05-19 16:22:08.544472",
"O_ID": 143,
"O_OL_CNT": 10,
"O_W_ID": 1
}x
“ORDERS”: {
“O_CUSTOMER_KEY”: “1.10.1938”):
"O_ALL_LOCAL": 1,
"O_CARRIER_ID": 2,
"O_C_ID": 1938,
"O_D_ID": 10,
"O_ENTRY_D": "2015-05-19 16:22:08.544472",
"O_ID": 1355,
"O_OL_CNT": 10,
"O_W_ID": 3
}
Document key: “1.10.1355”
©2015 Couchbase Inc. 53
Query Execution: Join
SELECT beer.NAME beername,
brewery.NAME brewery
FROM `beer-sample` beer
LEFT OUTER JOIN `beer-sample` brewery
ON KEYS beer.brewery_id
WHERE beer.type = 'beer'
AND brewery.type = 'brewery';
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 54
N1QL: Join
SELECT *
FROM ORDERS o INNER JOIN CUSTOMER c
ON KEYS (o.O_C_ID)
LEFT JOIN PREMIUM p
ON KEYS (c.C_PR_ID)
INNER JOIN demographics d
ON KEYS (c.c_DEMO_ID)
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Support INNER and LEFT OUTER joins
 Join order follows the order in the FROM clause.
 N1QL supports the nested loop joins now.
 Join is always from a key of one document(outer
table) to the document key of the second
document (inner table)
©2016 Couchbase Inc. 56
Covering Index
©2016 Couchbase Inc. 57
N1QL: Query Execution Flow
Clients
1. Submit the query over RESTAPI 8. Query result
2. Parse, Analyze, create Plan 7. Evaluate: Documents to results
3. Scan Request;
index filters
6. Fetch the documents
Index
Servic
e
Query
Service
Data
Servic
e
4. Get qualified doc keys
& index keys
5. Fetch Request,
doc keys
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
©2016 Couchbase Inc. 58
N1QL: covering index & a covered query
Clients
1. Submit the query over RESTAPI 6. Query result
2. Parse, Analyze, create Plan 5. Evaluate: Documents to results
3. Scan Request;
index filters
Index
Servic
e
Query
Service
4. Get qualified doc keys
& index keys
Create index i1 on CUSTOMER(c_id, c_max, c_first, c_last)
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
©2016 Couchbase Inc. 59
Inside a Query Service (without covering index)
Client
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
©2016 Couchbase Inc. 60
Inside a Query Service (with Covering index)
Client
Parse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
©2016 Couchbase Inc. 61
Covering indexes
 Reason for covering indexes is performance.
 Queries should get the benefit automatically.
 Index selection is still based on filters inWHERE clause only.
– Projected expressions are not considered for index selection.
– When there are multiple qualifying indices for the query, use USE INDEX hint to favor
covering index path.
 The covering index & expressions are noted in the explain.
 Only applies to Secondary Index scans
 Does not exploit the index ordering for ORDER BY… yet!
 Used in SELECT and INSERT INTO…SELECT only
 No path UPDATE, DELETE, MERGE (subqueries need USE KEYS)
©2016 Couchbase Inc. 62
create index idxstatecity on `beer-
sample`(state,city) using gsi;
EXPLAIN
SELECT city
FROM`beer-sample`
WHERE state = 'California';
N1QL: Example
{
"requestID": "eef73760-d09e-48e0-a43a-c8da1e0be998
"signature": "json",
"results": [
{
"#operator": "Sequence",
"~children": [
{
"#operator": "IndexScan",
"covers": [
"cover((meta(`beer-sample`).`id`))",
"cover((`beer-sample`.`state`))",
"cover((`beer-sample`.`city`))"
],
"index": "idxstatecity",
"keyspace": "beer-sample",
"namespace": "default",
"spans": [
{
]
©2016 Couchbase Inc. 63
create index idxstate on `beer-sample’(state) using gsi;
Create index idxstatecountry on `beer-sample(state, country) using gsi;
-- covered by idxstatecountry
select country from `beer-sample` where state = 'CA';
-- use directives to choose the optimal index…
select country from `beer-sample` use index (idxstatecountry) where state = 'CA';
-- Queries with expressions and aggregates can still use the covering index
explain select max(country) from `beer-sample` where state = 'CA';
explain select country || state
from `beer-sample` use index (idxstatecountry) where state = 'CA';
-- Even the expressions in order by will be covered
explain select country || state
from `beer-sample` use index (idxstatecountry)
where state = 'CA’
order by state;
N1QL: Examples
©2016 Couchbase Inc. 64
Create index idxstatecountry on `beer-sample(state, country) using gsi;
select country, max(state)
from `beer-sample` use index (idxstatecountry)
where state like '%' group by country;
-- UNION/ALL/INTERSECT/EXCEPT queries are supported
select country from `beer-sample` where state = 'CA'
UNION ALL
select country from `beer-sample` where state = 'Texas';
-- derived tables/subqueries
Select *
From (
select country from `beer-sample` where state = 'CA'
UNION ALL
select country from `beer-sample` where state = 'Texas’;
) as newtab;
N1QL: Examples
©2016 Couchbase Inc. 65
Create index idxstatecountry on `beer-sample(state, country) using gsi;
-- select in insert statement
insert into `travel-sample`(KEY k, value state)
Select country as k, state from `beer-sample b where stat = ‘ca’;
-- Supports Arrays
Create index idxarray on `beer-sample`(a, b);
Select b from `beer-sample` where a = [1, 2, 3, 4];
N1QL: Examples
©2016 Couchbase Inc. 66
Create index idxstatecountry on `beer-sample(state, country) using gsi;
-- Cannot use any index. Uses Primary scan.
select state, brewery from `beer-sample` where country = 'United States’;
— Uses index when appropriate.
Select state from `beer-sample` where state = 'CA'
UNION ALL
Select state from `travel-sample` where state = 'CA';
-- Uses index
select b.state, t.beer
from `beer-sample` as b inner join `travel-sample` as t
on keys (b.state)
where b.state = ‘ca’;
N1QL: Examples
©2016 Couchbase Inc. 67
FAQ
 Supports both GSI andVIEW indexes
 Supports partial indexes as well (create index withWHERE clause)
 Supports prepared statements
 Supports using key-only scan in any "query block" within a complex
statement.
– E.g. UNION/UNION-ALL/INTERSECT/EXCEPT and derived table queries
– Essentially, whenever a decision is made on the access path,
– covering-index-path is one of the options.
 Push the LIMIT+OFFSET to Index scan & data fetch
– SELECT state, country from `beer-sample` where state like ‘%’ order by state,
country LIMIT 10 OFFSET 20;
©2016 Couchbase Inc. 68
Summary: SQL & N1QL
Query Features SQL N1QL
Statements
 SELECT, INSERT, UPDATE, DELETE,
MERGE
 SELECT, INSERT, UPDATE, DELETE,
MERGE
Query Operations
 Select, Join, Project, Subqueries
 Strict Schema
 StrictType checking
 Select, Join, Project, Subqueries
 Nest & Unnest
 Look Ma! NoType Mismatch Errors!
 JSON keys act as columns
Schema  Predetermined Columns
 Fully addressable JSON
 Flexible document structure
DataTypes
 SQL Data types
 Conversion Functions
 JSON Data types
 Conversion Functions
Query Processing
 INPUT: Sets ofTuples
 OUPUT: Set ofTuples
 INPUT: Sets of JSON
 OUTPUT: Set of JSON
©2016 Couchbase Inc. 69
Summary
Summary
©2016 Couchbase Inc. 71
Query Changes the Possibilities for NoSQL
©2016 Couchbase Inc. 72

Más contenido relacionado

La actualidad más candente

Hive 3 a new horizon
Hive 3  a new horizonHive 3  a new horizon
Hive 3 a new horizonArtem Ervits
 
Azure Key Vault Integration in Scala
Azure Key Vault Integration in ScalaAzure Key Vault Integration in Scala
Azure Key Vault Integration in ScalaBraja Krishna Das
 
Better than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseBetter than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseAltinity Ltd
 
A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides Altinity Ltd
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j InternalsTobias Lindaaker
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelinesSumant Tambe
 
Column oriented database
Column oriented databaseColumn oriented database
Column oriented databaseKanike Krishna
 
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...HostedbyConfluent
 
Introduction of Apache Camel
Introduction of Apache CamelIntroduction of Apache Camel
Introduction of Apache CamelKnoldus Inc.
 
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBase
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBaseHBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBase
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBaseCloudera, Inc.
 
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureQConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureChris Richardson
 
How is Kafka so Fast?
How is Kafka so Fast?How is Kafka so Fast?
How is Kafka so Fast?Ricardo Paiva
 
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations
re:Invent 2022  DAT326 Deep dive into Amazon Aurora and its innovationsre:Invent 2022  DAT326 Deep dive into Amazon Aurora and its innovations
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovationsGrant McAlister
 
Lightweight Transactions in Scylla versus Apache Cassandra
Lightweight Transactions in Scylla versus Apache CassandraLightweight Transactions in Scylla versus Apache Cassandra
Lightweight Transactions in Scylla versus Apache CassandraScyllaDB
 
9. Object Relational Databases in DBMS
9. Object Relational Databases in DBMS9. Object Relational Databases in DBMS
9. Object Relational Databases in DBMSkoolkampus
 
Breakthrough OLAP performance with Cassandra and Spark
Breakthrough OLAP performance with Cassandra and SparkBreakthrough OLAP performance with Cassandra and Spark
Breakthrough OLAP performance with Cassandra and SparkEvan Chan
 

La actualidad más candente (20)

Hive 3 a new horizon
Hive 3  a new horizonHive 3  a new horizon
Hive 3 a new horizon
 
Azure Key Vault Integration in Scala
Azure Key Vault Integration in ScalaAzure Key Vault Integration in Scala
Azure Key Vault Integration in Scala
 
Better than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseBetter than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouse
 
A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
 
Tuning kafka pipelines
Tuning kafka pipelinesTuning kafka pipelines
Tuning kafka pipelines
 
Column oriented database
Column oriented databaseColumn oriented database
Column oriented database
 
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
 
HBase Storage Internals
HBase Storage InternalsHBase Storage Internals
HBase Storage Internals
 
Introduction of Apache Camel
Introduction of Apache CamelIntroduction of Apache Camel
Introduction of Apache Camel
 
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBase
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBaseHBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBase
HBaseCon 2013: Apache HBase and HDFS - Understanding Filesystem Usage in HBase
 
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureQConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
 
How is Kafka so Fast?
How is Kafka so Fast?How is Kafka so Fast?
How is Kafka so Fast?
 
Deep Dive on Amazon Redshift
Deep Dive on Amazon RedshiftDeep Dive on Amazon Redshift
Deep Dive on Amazon Redshift
 
Pregel and giraph
Pregel and giraphPregel and giraph
Pregel and giraph
 
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations
re:Invent 2022  DAT326 Deep dive into Amazon Aurora and its innovationsre:Invent 2022  DAT326 Deep dive into Amazon Aurora and its innovations
re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations
 
Lightweight Transactions in Scylla versus Apache Cassandra
Lightweight Transactions in Scylla versus Apache CassandraLightweight Transactions in Scylla versus Apache Cassandra
Lightweight Transactions in Scylla versus Apache Cassandra
 
9. Object Relational Databases in DBMS
9. Object Relational Databases in DBMS9. Object Relational Databases in DBMS
9. Object Relational Databases in DBMS
 
(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive
 
Breakthrough OLAP performance with Cassandra and Spark
Breakthrough OLAP performance with Cassandra and SparkBreakthrough OLAP performance with Cassandra and Spark
Breakthrough OLAP performance with Cassandra and Spark
 

Destacado

N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.Keshav Murthy
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseCecile Le Pape
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingKeshav Murthy
 
Understanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesUnderstanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesKeshav Murthy
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLKeshav Murthy
 
Tuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesTuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesKeshav Murthy
 
Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Keshav Murthy
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Cecile Le Pape
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedKorea Sdec
 
ORM, JPA, & Hibernate Overview
ORM, JPA, & Hibernate OverviewORM, JPA, & Hibernate Overview
ORM, JPA, & Hibernate OverviewBrett Meyer
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Keshav Murthy
 

Destacado (13)

N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
 
Drilling on JSON
Drilling on JSONDrilling on JSON
Drilling on JSON
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and Indexing
 
Understanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesUnderstanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune Queries
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
 
Tuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesTuning for Performance: indexes & Queries
Tuning for Performance: indexes & Queries
 
Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.Deep dive into N1QL: SQL for JSON: Internals and power features.
Deep dive into N1QL: SQL for JSON: Internals and power features.
 
Couchbase Day
Couchbase DayCouchbase Day
Couchbase Day
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speed
 
ORM, JPA, & Hibernate Overview
ORM, JPA, & Hibernate OverviewORM, JPA, & Hibernate Overview
ORM, JPA, & Hibernate Overview
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data.
 

Similar a Query in Couchbase. N1QL: SQL for JSON

Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Matthew Groves
 
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it tooQuerying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it tooAll Things Open
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications Keshav Murthy
 
Querying NoSQL with SQL - MIGANG - July 2017
Querying NoSQL with SQL - MIGANG - July 2017Querying NoSQL with SQL - MIGANG - July 2017
Querying NoSQL with SQL - MIGANG - July 2017Matthew Groves
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONKeshav Murthy
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Keshav Murthy
 
JSON Data Modeling - GDG Indy - April 2020
JSON Data Modeling - GDG Indy - April 2020JSON Data Modeling - GDG Indy - April 2020
JSON Data Modeling - GDG Indy - April 2020Matthew Groves
 
Json data modeling june 2017 - pittsburgh tech fest
Json data modeling   june 2017 - pittsburgh tech festJson data modeling   june 2017 - pittsburgh tech fest
Json data modeling june 2017 - pittsburgh tech festMatthew Groves
 
Putting the SQL Back in NoSQL - October 2022 - All Things Open
Putting the SQL Back in NoSQL - October 2022 - All Things OpenPutting the SQL Back in NoSQL - October 2022 - All Things Open
Putting the SQL Back in NoSQL - October 2022 - All Things OpenMatthew Groves
 
NoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseNoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseBrant Burnett
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5Keshav Murthy
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0Keshav Murthy
 
Data Modeling and Relational to NoSQL
 Data Modeling and Relational to NoSQL  Data Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQL DATAVERSITY
 
JSON Data Modeling - July 2018 - Tulsa Techfest
JSON Data Modeling - July 2018 - Tulsa TechfestJSON Data Modeling - July 2018 - Tulsa Techfest
JSON Data Modeling - July 2018 - Tulsa TechfestMatthew Groves
 
Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...EDB
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revisedMongoDB
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessMongoDB
 
OSCON 2011 CouchApps
OSCON 2011 CouchAppsOSCON 2011 CouchApps
OSCON 2011 CouchAppsBradley Holt
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB
 
SQL Access to NoSQL
SQL Access to NoSQLSQL Access to NoSQL
SQL Access to NoSQLProgress
 

Similar a Query in Couchbase. N1QL: SQL for JSON (20)

Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017Querying NoSQL with SQL - KCDC - August 2017
Querying NoSQL with SQL - KCDC - August 2017
 
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it tooQuerying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too
Querying NoSQL with SQL: HAVING Your JSON Cake and SELECTing it too
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
 
Querying NoSQL with SQL - MIGANG - July 2017
Querying NoSQL with SQL - MIGANG - July 2017Querying NoSQL with SQL - MIGANG - July 2017
Querying NoSQL with SQL - MIGANG - July 2017
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSON
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
 
JSON Data Modeling - GDG Indy - April 2020
JSON Data Modeling - GDG Indy - April 2020JSON Data Modeling - GDG Indy - April 2020
JSON Data Modeling - GDG Indy - April 2020
 
Json data modeling june 2017 - pittsburgh tech fest
Json data modeling   june 2017 - pittsburgh tech festJson data modeling   june 2017 - pittsburgh tech fest
Json data modeling june 2017 - pittsburgh tech fest
 
Putting the SQL Back in NoSQL - October 2022 - All Things Open
Putting the SQL Back in NoSQL - October 2022 - All Things OpenPutting the SQL Back in NoSQL - October 2022 - All Things Open
Putting the SQL Back in NoSQL - October 2022 - All Things Open
 
NoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseNoSQL Data Modeling using Couchbase
NoSQL Data Modeling using Couchbase
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0
 
Data Modeling and Relational to NoSQL
 Data Modeling and Relational to NoSQL  Data Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQL
 
JSON Data Modeling - July 2018 - Tulsa Techfest
JSON Data Modeling - July 2018 - Tulsa TechfestJSON Data Modeling - July 2018 - Tulsa Techfest
JSON Data Modeling - July 2018 - Tulsa Techfest
 
Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revised
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational Awareness
 
OSCON 2011 CouchApps
OSCON 2011 CouchAppsOSCON 2011 CouchApps
OSCON 2011 CouchApps
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch Introduction
 
SQL Access to NoSQL
SQL Access to NoSQLSQL Access to NoSQL
SQL Access to NoSQL
 

Más de Keshav Murthy

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0Keshav Murthy
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...Keshav Murthy
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresKeshav Murthy
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliKeshav Murthy
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Keshav Murthy
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber Keshav Murthy
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersKeshav Murthy
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorKeshav Murthy
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONKeshav Murthy
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Keshav Murthy
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLKeshav Murthy
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixKeshav Murthy
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherKeshav Murthy
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Keshav Murthy
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013Keshav Murthy
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveKeshav Murthy
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Keshav Murthy
 

Más de Keshav Murthy (17)

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing features
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developers
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index Advisor
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSON
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QL
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on Informix
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all together
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep dive
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
 

Último

Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Modelsaagamshah0812
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdfWave PLM
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendTest Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendArshad QA
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️Delhi Call girls
 
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsHow To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsAndolasoft Inc
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about usDynamic Netsoft
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...OnePlan Solutions
 
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfActive Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfCionsystems
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providermohitmore19
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
Clustering techniques data mining book ....
Clustering techniques data mining book ....Clustering techniques data mining book ....
Clustering techniques data mining book ....ShaimaaMohamedGalal
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxbodapatigopi8531
 

Último (20)

Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf
 
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendTest Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and Backend
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsHow To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.js
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about us
 
Exploring iOS App Development: Simplifying the Process
Exploring iOS App Development: Simplifying the ProcessExploring iOS App Development: Simplifying the Process
Exploring iOS App Development: Simplifying the Process
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
 
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfActive Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdf
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
Clustering techniques data mining book ....
Clustering techniques data mining book ....Clustering techniques data mining book ....
Clustering techniques data mining book ....
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
 

Query in Couchbase. N1QL: SQL for JSON

  • 3. ©2016 Couchbase Inc. 3 Agenda Thinking in N1QL Anatomy of Query
  • 5. ©2016 Couchbase Inc. 5 Business Application Questions &Tasks How many new customers we got last month? Get the list of stores in customer region Search stores for the shoe customer is looking for? Generate a list of shipment due today Load the new inventory data Update the sale prices in outlet stores only Merge the customer lists
  • 6. ©2016 Couchbase Inc. 6 NoSQL Landscape Document • Couchbase • MongoDB • DynamoDB • DocumentDB Graph • OrientDB • Neo4J • DEX • GraphBase Key-Value • Couchbase • Riak • BerkeleyDB • Redis • … Wide Column • Hbase • Cassandra • Hypertable JSON to Represent Data
  • 7. ©2015 Couchbase Inc. 7 Data Management Landscape Processing in Files MapReduce Generic fileformats Rows/Columns in files (tables) Hive – Pig - etc Query Impala Hive NoSQL MongoDB Couchbase Hbase Cassandra HADOOP (Analytical) Disk & Storage Highly Structured Data SQL, R, etc Bytes & Blocks $100K – $200K /TB$1K/TB$10K/TB Semi Structured & Self describingNo Structure OLTP EDW $10K-$20K/TB Drill Operational Big data Couchbase 4
  • 8. ©2016 Couchbase Inc. 8 Properties of Real-World Data  Rich structure – Attributes, Sub-structure  Relationships – To other data  Value evolution – Data is updated  Structure evolution – Data is reshaped Customer Name DOB Billing Connections Purchases
  • 9. ©2016 Couchbase Inc. 9 Transform: Relational to JSON { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } DocumentKey: CBL2015 CustomerID Name DOB CBL2015 Jane Smith 1990-01-30 Customer ID Type Cardnum Expiry CBL2015 visa 5827… 2019-03 CBL2015 maste r 6274… 2018-12 CustomerID ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith CustomerID item amt CBL2015 mac 2823.52 CBL2015 ipad2 623.52 CustomerID ConnId Name CBL2015 XYZ987 Joe Smith CBL2015 SKR007 Sam Smith Contacts Customer Billing ConnectionsPurchases
  • 10. ©2016 Couchbase Inc. 10 JSON { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] }  JSON is a means to the end and not the end itself – JSON is the representation of the enterprise data model for applications – JSON flexibility translates to application flexibility • Simple flattened data can be represented • Entities with complex data, always accessed analyzed together should belong together – Applications are designed to handle the flexible data model.
  • 11. ©2016 Couchbase Inc. 11 Models for Representing Data Data Concern Relational Model JSON Document Model (NoSQL) Rich Structure  Multiple flat tables  Constant assembly / disassembly  Documents  No assembly required! Relationships  Represented  Queried (SQL)  Represented  Queried? Not until now… Value Evolution  Data can be updated  Data can be updated Structure Evolution  Uniform and rigid  Manual change (disruptive)  Flexible  Dynamic change
  • 12. ©2016 Couchbase Inc. 12 ResultSet
  • 13. ©2016 Couchbase Inc. 13 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER Built Manually; Expensive
  • 14. ©2016 Couchbase Inc. 14 { "Name" : "Jane Smith", "DOB" : "1990-01-30", "Billing" : [ { "type" : "visa", "cardnum" : "5827-2842-2847-3909", "expiry" : "2019-03" }, { "type" : "master", "cardnum" : "6274-2842-2847-3909", "expiry" : "2019-03" } ], "Connections" : [ { "CustId" : "XYZ987", "Name" : "Joe Smith" }, { "CustId" : "PQR823", "Name" : "Dylan Smith" } { "CustId" : "PQR823", "Name" : "Dylan Smith" } ], "Purchases" : [ { "id":12, item: "mac", "amt": 2823.52 } { "id":19, item: "ipad2", "amt": 623.52 } ] } LoyaltyInfo ResultDocuments Orders CUSTOMER
  • 15. ©2016 Couchbase Inc. 15 N1QL Changes the Possibilities for NoSQL
  • 16. ©2016 Couchbase Inc. 16 Goal of N1QL: SQL for JSON Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  • 17. ©2016 Couchbase Inc. 17 N1QL: Developers & Enterprises Application Developers in all languages –Couchbase SDK Support for N1QL –Open REST API Exchanges data with other databases using Standard Tools Simba provides ODBC, JDBC drivers Tableau, INFA interoperability
  • 18. ©2016 Couchbase Inc. 18 N1QL: expressive  Access to every part of JSON document  Scalar & Aggreate functions  Issue subquery in any expressions  Subqueries  Subqueries in the FROM clause
  • 19. ©2016 Couchbase Inc. 19 N1QL: powerful  Access to every part of JSON document  JOINS, Aggregations, standard scalar functions  Aggregation on arrays  NEST & UNNEST operations  Covering Index
  • 20. ©2016 Couchbase Inc. 20 N1QL: querying  INSERT  UPDATE  DELETE  MERGE  SELECT  EXPLAIN
  • 21. ©2016 Couchbase Inc. 21 N1QL: transforming & manipulating  FullTransformation of the data via Query.  INSERT – INSERT single & multiple documents – INSERT result a SELECT statement  DELETE documents based on complex filter  UPDATE any part of JSON document & use complex filter.  MERGE two sets of documents using traditional MERGE statement
  • 23. ©2016 Couchbase Inc. 23 Where’s My Beer? SELECT {"Mybeer": "My Beer is " || beer.name || "."}, ARRAY_AGG({"name":brewery.name}) brewery, ARRAY_AGG({"name":brewery.name, "state":brewery.state, "city":brewery.city, "location":brewery.geo}) locations, ARRAY_COUNT(ARRAY_AGG(brewery.name)) as brewery_count FROM `beer-sample` beer LEFT OUTER JOIN `beer-sample` brewery ON KEYS beer.brewery_id WHERE beer.type = 'beer' AND brewery.type = 'brewery' AND brewery.state = 'California' GROUP BY beer.name ORDER BY ARRAY_COUNT(ARRAY_AGG(brewery.name)) desc, beer.name asc LIMIT 5 ;
  • 24. ©2016 Couchbase Inc. 24 Hands-onWorkshop  Execute the query1  Observe & discuss the structure of the document returned by the query  Run the EXPLAIN for the query  Modify – WHERE clause – LIMIT clause – ORDER BY clause
  • 25. ©2016 Couchbase Inc. 25 QUERY EXECUTION
  • 26. ©2016 Couchbase Inc. 26 N1QL: Query Execution Flow Clients 1. Submit the query over RESTAPI 8. Query result 2. Parse, Analyze, create Plan 7. Evaluate: Documents to results 3. Scan Request; index filters 6. Fetch the documents Index Servic e Query Service Data Servic e 4. Get qualified doc keys 5. Fetch Request, doc keys SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 27. ©2016 Couchbase Inc. 27 Inside a Query Service Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  • 28. ©2016 Couchbase Inc. 28 Inside a Query Service FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e Prepare Phase Projection Phase Run Phase Scan Phase Fetch Phase Sort Phase
  • 29. ©2016 Couchbase Inc. 29 QUERY PREPARE
  • 30. ©2016 Couchbase Inc. 30 Prepare Phase: Parse & Semantic Check  Analyzes the Query for syntax & grammar  Only verifies for existence of referenced buckets  Flexible schema means, you can refer to arbitrary attribute names. N1QL key names are CaSe SENSitive.  A key is NULL if it’s a known null. {“location”:null}  A key is MISSING if not present in the document  Use IS NULL and IS MISSING to check for each condition  Full reference to JSON structure – Nested reference: CUSTOMER.contact.address.state – Array Reference: CUSTOMER.c_contact.phone_number[0]  SQL is enhanced to access & manipulateArrays Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 31. ©2016 Couchbase Inc. 31 Prepare: PLAN Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Each query can be executed in several ways  Create the query execution plan – Access path for each keyspace reference – Decide on the filters to push down – Determine Join order and join method – Create the execution tree  For each keyspace reference: – Look at the available indices – Match the filters in the query with index keys – Choose one or more indices for each keyspace
  • 32. ©2016 Couchbase Inc. 32 PREPARE Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Ever statement executed in the engine is PREPARED  When you have same statement executed millions of times, you can save prepare time  Prepare statement can use simple adhoc statement PREPARE p1 FROM SELECT * from `beer-sample` LIMIT 1; EXECUTE p1;  Compare the execution time  Prepare Statement can exploit parameters to bind PREPARE p2 FROM SELECT * from `beer-sample` WHERE abv = $1; $ curl http://localhost:8093/query/service -H "Content- Type: application/json" -d '{"prepared":"p2", "args":[5.0]}’ $ curl http://localhost:8093/query/service -d 'prepared ="mp"&args=[5.0]’
  • 33. ©2016 Couchbase Inc. 33 PREPARE Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Ever statement executed in the engine is PREPARED  When you have same statement executed millions of times, you can save prepare time  Prepare statement can use simple adhoc statement PREPARE p1 FROM SELECT * from `beer-sample` LIMIT 1; EXECUTE p1;  Compare the execution time  Prepare Statement can exploit parameters to bind PREPARE p2 FROM SELECT * from `beer-sample` WHERE abv = $1; $ curl http://localhost:8093/query/service -H "Content- Type: application/json" -d '{"prepared":"p2", "args":[5.0]}’ $ curl http://localhost:8093/query/service -d 'prepared ="mp"&args=[5.0]’
  • 34. ©2016 Couchbase Inc. 34 PREPARE Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Parameter can be named or positional.  Named parameters: curl http://localhost:8093/query/service -d 'statement=prepare p3 from select count(*) from `beer-sample` where abv > $val’ curl http://localhost:8093/query/service -H "Content-Type: application/json" -d '{"prepared":"p3", "$val":5.0}’ curl http://localhost:8093/query/service -d 'prepared="p3"&$val=5.0'
  • 35. ©2016 Couchbase Inc. 35 PREPARE: Cluster Environment Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan curl http://localhost:8093/query/service -d 'statement=prepare p4 from select name from `beer-sample` where name LIKE $val’ curl http://localhost:8093/query/service -H "Content-Type: application/json" -d '{"prepared":"p4", "encoded_plan":"H4sIAAAJbogA/5xSTW/qMBD8K5F5B5B4T3oSp9x6KBJ qD0gcq4q4yZAYHNtdOxSK0t/edaDlq5dys2dnZ2fW3gmY3BYo5k5LI1Ihhs LIGnxyIz5bB5LBkkh3one8iBleG+4EUz7ySumCwN1P56S7JlSW1HtkdaSbV KakaknbWc7+hkKZAhtGe24PM7TC1juZR88vAP31snYahyBflQIL2ejAaOOV KRkpvRLt8GKWJKk19LehmxyPEfLqt8YurYyVDqC4OWsKFZSNr9PvZydK2b8 sKmWDRKsVkj9rqQdXOhPDzVJPyS6Rx/wEz/PmLF77zjo2LhJ/lhbt87Uzc9 TjcmQIfo610ijh48oOodLT2On/9lJpFgiy7kYIr0ojQ0OI/YcvuPScmmsBm xB/JMFJQuJGyYJsnXhotpBE8h44S5C8VWByV32cPNx3+xHtZwAAAP//FkDT W/ICAAA=", "$val":"%Ale%"}'
  • 36. ©2016 Couchbase Inc. 36 PREPARE: Summary Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Using prepare state saves time. – Amount of saving depends on the query complexity, index choices.  Effective when the same statement or statement with parameters are executed  Encoded plan enables SDKs and applications to prepare once and use it on any node in the cluster.  SDKs use prepared statement behind the scenes.
  • 37. ©2016 Couchbase Inc. 37 Hands-onWorkshop  Prepare the queries P1 through P4.  Execute them in cbq and via curl  Compare the execution timings.
  • 38. ©2016 Couchbase Inc. 38 QUERY PLANNING & INDEX SELECTION
  • 39. ©2016 Couchbase Inc. 39 PLAN:We’re not done yet! Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Each query can be executed in several ways  Create the query execution plan – Access path for each keyspace reference – Decide on the filters to push down – Determine Join order and join method – Create the execution tree  For each keyspace reference: – Look at the available indices – Match the filters in the query with index keys – Choose one or more indices for each keyspace
  • 40. ©2016 Couchbase Inc. 40 Index Overview: Primary Index  Primary Index  CREATE PRIMARY INDEX `beer-sample`;  Document key is unique for the bucket.  Primary index is used when no other qualifying index is available or when no predicate is given in the query.  PrimaryScan is equivalent of full table scan "beer-sample": { "abv": 7.2, "brewery_id": "21st_amendment_brewery_cafe", "category": "North American Ale", "description": "Deep golden color...!", "ibu": 0, "name": "21A IPA", "srm": 0, "style": "American-Style India Pale Ale", "type": "beer", "upc": 0, "updated": "2010-07-22 20:00:20" } Document key: “guiness_d1”
  • 41. ©2016 Couchbase Inc. 41 "beer-sample": { "abv": 7.2, "brewery_id": "21st_amendment_brewery_cafe", "category": "North American Ale", "description": "Deep golden color...!", "ibu": 0, "name": "21A IPA", "srm": 0, "style": "American-Style India Pale Ale", "type": "beer", "upc": 0, "updated": "2010-07-22 20:00:20" } Document key: “Pale Ale.A1” Index Overview: Secondary Index Secondary Index can be created on any combination of attribute names. create index idxbeerabv on `beer- sample`(abv, state); create index idxtypeabvstate on `beer-sample`(abv, state) where type = 'beer';  Need to have matching indices with right key- ordering
  • 42. ©2016 Couchbase Inc. 42 GSI Index: Key details  SupportedTypes  String, Boolean, Numeric, Nil,Array, Sub-document  Comp  Total length of the keys  4 KB – actual length of the key indexed  Number of keys  4096!
  • 43. ©2016 Couchbase Inc. 43 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan create index idxbeerabv on `beer-sample`(abv, state); EXPLAIN SELECT count(*) FROM `beer-sample` WHERE abv > 5.0 AND state = 'California'; select * from system:indexes where keyspace_id = 'beer-sample';  Explain provides the JSON representation of the query plan  Focus on the index selection and the predicates pushed down
  • 44. ©2016 Couchbase Inc. 44 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan create index idxtypeabvstate on `beer- sample`(abv, state) where type = ‘beer’; select * from system:indexes where name = 'idxtypeabvstate'; "indexes": { "condition": "(`type` = "beer")", "datastore_id": "http://127.0.0.1:8091", "id": "611ffca1720b7868", "index_key": [ "`abv`", "`state`" ], "keyspace_id": "beer-sample", "name": "idxtypeabvstate", "namespace_id": "default", "state": "online", "using": "gsi" }
  • 45. ©2016 Couchbase Inc. 45 Plan: Index selection – Partial Index Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan create index idxabvp1 on `beer-sample`(abv, state) where type = 'beer' and abv > 5.0; explain select count(*) from `beer-sample` where type = 'beer' and abv = 7.0 ;
  • 46. ©2015 Couchbase Inc. 46 Power Features: IntersectScan (Multi-Index Scan) create index idxbeerabv on `beer-sample`(abv); create index idxbeercat on `beer-sample`(category); select * from `beer-sample` where abv > 15 and category like '%North%';  IntersectScan using multiple indices: – Multiple indices are scanned in parallel – Provides more flexibility in using the indices for filters – Requires less number of indexes defined on table. • Can save on disk space and memory space as well.
  • 47. ©2016 Couchbase Inc. 47 IntersectScan method using multiple indices create index idxbeerabv on `beer-sample`(abv); create index idxbeercat on `beer-sample`(category); select * from `beer-sample` where abv > 15 and category like '%North%';  IntersectScan using multiple indices: – Multiple indices are scanned in parallel – Provides more flexibility in using the indices for filters – Requires less number of indexes defined on table. • Can save on disk space and memory space as well.
  • 48. ©2016 Couchbase Inc. 48 Plan Directives: USE INDEX create index idxbeerabv on `beer-sample`(abv); create index idxbeercat on `beer-sample`(category); select * from `beer-sample` USE INDEX (idxbeerabv) where abv > 15 and category like '%North%'; select * from `beer-sample` USE INDEX (idxbeerabv, idxbeercat) where abv > 15 and category like '%North%’;  USE INDEX suggests one or more appropriate indices  Optimizer validates the indices. Primary Index is the backup index.  Provides stability to the optimizer plan.
  • 49. ©2016 Couchbase Inc. 49 Query Execution: Fetch  List of qualified document-keys are grouped into batches.  List of the documents is obtained from the Index or specified directly via USE KEYS clause.  Fetch request is done in parallel.  The join operation use the fetch operation to get the matching document.  Fetch results are streamed into next operators.  For big queries, scan-fetch-join-filter-aggregation will be executing in parallel. Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 51. ©2016 Couchbase Inc. 51 Query Execution: Join  You can join any two key spaces if one has document-key of the other.  You can store multiple entities within the same bucket and join between distinct groups  Uses Nested Loop JOIN now  JOINs are done in the same order specified in the query  Index selection is important for the first keyspace in the FROM clause.  Qualified documents from that scan is joined with the other Keyspace using the DOCUMENT KEYS Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 52. ©2015 Couchbase Inc. 52 Query Execution: Join "CUSTOMER": { "C_D_ID": 10, "C_ID": 1938, "C_W_ID": 1, "C_BALANCE": -10, "C_CITY": ”San Jose", "C_CREDIT": "GC”, "C_DELIVERY_CNT": 0, "C_DISCOUNT": 0.3866, "C_FIRST": ”Jay", "C_LAST": ”Smith", "C_MIDDLE": "OE", "C_PAYMENT_CNT": 1, "C_PHONE": ”555-123-1234", "C_SINCE": "2015-03-22 00:50:42.822518", "C_STATE": ”CA", "C_STREET_1": ”555, Tideway Drive", "C_STREET_2": ”Alameda", "C_YTD_PAYMENT": 10, "C_ZIP": ”94501" } Document key: “1.10.1938” Document key: “1.10.143” “ORDERS”: { “O_CUSTOMER_KEY”: “1.10.1938): "O_D_ID": 10, "O_ID": 1, "O_ALL_LOCAL": 1, "O_CARRIER_ID": 2, "O_C_ID": 1938, "O_ENTRY_D": "2015-05-19 16:22:08.544472", "O_ID": 143, "O_OL_CNT": 10, "O_W_ID": 1 }x “ORDERS”: { “O_CUSTOMER_KEY”: “1.10.1938”): "O_ALL_LOCAL": 1, "O_CARRIER_ID": 2, "O_C_ID": 1938, "O_D_ID": 10, "O_ENTRY_D": "2015-05-19 16:22:08.544472", "O_ID": 1355, "O_OL_CNT": 10, "O_W_ID": 3 } Document key: “1.10.1355”
  • 53. ©2015 Couchbase Inc. 53 Query Execution: Join SELECT beer.NAME beername, brewery.NAME brewery FROM `beer-sample` beer LEFT OUTER JOIN `beer-sample` brewery ON KEYS beer.brewery_id WHERE beer.type = 'beer' AND brewery.type = 'brewery'; Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 54. ©2015 Couchbase Inc. 54 N1QL: Join SELECT * FROM ORDERS o INNER JOIN CUSTOMER c ON KEYS (o.O_C_ID) LEFT JOIN PREMIUM p ON KEYS (c.C_PR_ID) INNER JOIN demographics d ON KEYS (c.c_DEMO_ID) Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Support INNER and LEFT OUTER joins  Join order follows the order in the FROM clause.  N1QL supports the nested loop joins now.  Join is always from a key of one document(outer table) to the document key of the second document (inner table)
  • 55. ©2016 Couchbase Inc. 56 Covering Index
  • 56. ©2016 Couchbase Inc. 57 N1QL: Query Execution Flow Clients 1. Submit the query over RESTAPI 8. Query result 2. Parse, Analyze, create Plan 7. Evaluate: Documents to results 3. Scan Request; index filters 6. Fetch the documents Index Servic e Query Service Data Servic e 4. Get qualified doc keys & index keys 5. Fetch Request, doc keys SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 57. ©2016 Couchbase Inc. 58 N1QL: covering index & a covered query Clients 1. Submit the query over RESTAPI 6. Query result 2. Parse, Analyze, create Plan 5. Evaluate: Documents to results 3. Scan Request; index filters Index Servic e Query Service 4. Get qualified doc keys & index keys Create index i1 on CUSTOMER(c_id, c_max, c_first, c_last) SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 58. ©2016 Couchbase Inc. 59 Inside a Query Service (without covering index) Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  • 59. ©2016 Couchbase Inc. 60 Inside a Query Service (with Covering index) Client Parse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  • 60. ©2016 Couchbase Inc. 61 Covering indexes  Reason for covering indexes is performance.  Queries should get the benefit automatically.  Index selection is still based on filters inWHERE clause only. – Projected expressions are not considered for index selection. – When there are multiple qualifying indices for the query, use USE INDEX hint to favor covering index path.  The covering index & expressions are noted in the explain.  Only applies to Secondary Index scans  Does not exploit the index ordering for ORDER BY… yet!  Used in SELECT and INSERT INTO…SELECT only  No path UPDATE, DELETE, MERGE (subqueries need USE KEYS)
  • 61. ©2016 Couchbase Inc. 62 create index idxstatecity on `beer- sample`(state,city) using gsi; EXPLAIN SELECT city FROM`beer-sample` WHERE state = 'California'; N1QL: Example { "requestID": "eef73760-d09e-48e0-a43a-c8da1e0be998 "signature": "json", "results": [ { "#operator": "Sequence", "~children": [ { "#operator": "IndexScan", "covers": [ "cover((meta(`beer-sample`).`id`))", "cover((`beer-sample`.`state`))", "cover((`beer-sample`.`city`))" ], "index": "idxstatecity", "keyspace": "beer-sample", "namespace": "default", "spans": [ { ]
  • 62. ©2016 Couchbase Inc. 63 create index idxstate on `beer-sample’(state) using gsi; Create index idxstatecountry on `beer-sample(state, country) using gsi; -- covered by idxstatecountry select country from `beer-sample` where state = 'CA'; -- use directives to choose the optimal index… select country from `beer-sample` use index (idxstatecountry) where state = 'CA'; -- Queries with expressions and aggregates can still use the covering index explain select max(country) from `beer-sample` where state = 'CA'; explain select country || state from `beer-sample` use index (idxstatecountry) where state = 'CA'; -- Even the expressions in order by will be covered explain select country || state from `beer-sample` use index (idxstatecountry) where state = 'CA’ order by state; N1QL: Examples
  • 63. ©2016 Couchbase Inc. 64 Create index idxstatecountry on `beer-sample(state, country) using gsi; select country, max(state) from `beer-sample` use index (idxstatecountry) where state like '%' group by country; -- UNION/ALL/INTERSECT/EXCEPT queries are supported select country from `beer-sample` where state = 'CA' UNION ALL select country from `beer-sample` where state = 'Texas'; -- derived tables/subqueries Select * From ( select country from `beer-sample` where state = 'CA' UNION ALL select country from `beer-sample` where state = 'Texas’; ) as newtab; N1QL: Examples
  • 64. ©2016 Couchbase Inc. 65 Create index idxstatecountry on `beer-sample(state, country) using gsi; -- select in insert statement insert into `travel-sample`(KEY k, value state) Select country as k, state from `beer-sample b where stat = ‘ca’; -- Supports Arrays Create index idxarray on `beer-sample`(a, b); Select b from `beer-sample` where a = [1, 2, 3, 4]; N1QL: Examples
  • 65. ©2016 Couchbase Inc. 66 Create index idxstatecountry on `beer-sample(state, country) using gsi; -- Cannot use any index. Uses Primary scan. select state, brewery from `beer-sample` where country = 'United States’; — Uses index when appropriate. Select state from `beer-sample` where state = 'CA' UNION ALL Select state from `travel-sample` where state = 'CA'; -- Uses index select b.state, t.beer from `beer-sample` as b inner join `travel-sample` as t on keys (b.state) where b.state = ‘ca’; N1QL: Examples
  • 66. ©2016 Couchbase Inc. 67 FAQ  Supports both GSI andVIEW indexes  Supports partial indexes as well (create index withWHERE clause)  Supports prepared statements  Supports using key-only scan in any "query block" within a complex statement. – E.g. UNION/UNION-ALL/INTERSECT/EXCEPT and derived table queries – Essentially, whenever a decision is made on the access path, – covering-index-path is one of the options.  Push the LIMIT+OFFSET to Index scan & data fetch – SELECT state, country from `beer-sample` where state like ‘%’ order by state, country LIMIT 10 OFFSET 20;
  • 67. ©2016 Couchbase Inc. 68 Summary: SQL & N1QL Query Features SQL N1QL Statements  SELECT, INSERT, UPDATE, DELETE, MERGE  SELECT, INSERT, UPDATE, DELETE, MERGE Query Operations  Select, Join, Project, Subqueries  Strict Schema  StrictType checking  Select, Join, Project, Subqueries  Nest & Unnest  Look Ma! NoType Mismatch Errors!  JSON keys act as columns Schema  Predetermined Columns  Fully addressable JSON  Flexible document structure DataTypes  SQL Data types  Conversion Functions  JSON Data types  Conversion Functions Query Processing  INPUT: Sets ofTuples  OUPUT: Set ofTuples  INPUT: Sets of JSON  OUTPUT: Set of JSON
  • 68. ©2016 Couchbase Inc. 69 Summary
  • 70. ©2016 Couchbase Inc. 71 Query Changes the Possibilities for NoSQL

Notas del editor

  1. NoSQL, although generally accepted as Not Only SQL, generally refers to databases which lack SQL. Implementing generally accepted subset of SQL for flexible data model on a distributed system IS HARD. Once the SQL language, transaction became optional, flurry of databases were created using distinct approaches for common use-cases. KEY-Value simply provided quick access to data for a given KEY. Lot of the databases in this group provide additional functionality compared to memcached. Wide Column databases Graph databases can store large number of arbitrary columns in each row Document databases aggregate data into a hierarchical structure. With JSON is a means to the end. Document databases provide flexible schema,built-in data types, rich structure, implicit relationships using JSON. With lot of these databases simple things like GET and PUT were done via very simple API. Anything compilicated requires long, client side programs. Let’s seen an example.
  2. Let’s look at modeling Customer data.
  3. SQL language operates on set of relations (table) and then projects another relation (table)
  4. N1QL is a SQL based language designed for JSON. Input is set of related sets of JSON documents. It operates on these documents and produces another set of JSON documents. All of the SQL operattions, select, join, project operations are supported. All of the common statements are supported: SELECT, INSERT, UPDATE, DELETE and MERGE. Additionally, N1QL extends the language to support: -- Fully address, access and modify any part of the JSON document. -- Handle the full flexible schema where schema is self-described by each document. -- key-value pairs could be missing, data types could change and structure could change!
  5. N1QL is a SQL based language designed for JSON. Input is set of related sets of JSON documents. It operates on these documents and produces another set of JSON documents. All of the SQL operattions, select, join, project operations are supported. All of the common statements are supported: SELECT, INSERT, UPDATE, DELETE and MERGE. Additionally, N1QL extends the language to support: -- Fully address, access and modify any part of the JSON document. -- Handle the full flexible schema where schema is self-described by each document. -- key-value pairs could be missing, data types could change and structure could change!
  6. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  7. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  8. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  9. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  10. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  11. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data.
  12. Data-parallel — Query latency scales up with cores Memory-bound
  13. Data-parallel — Query latency scales up with cores Memory-bound
  14. Global Secondary Indexes Tackles indexer for fast query execution with efficient index maintenance for N1QL Queries High Performance Indexing Projector and Router : Coordinate and communicate efficient index change notifications between data service and index service. Supervisor – Indexer and scanner Indexer : Maintain large number of indexes as change notifications arrive Scanner: Respond to Query Service index-scan requests with rich set of consistency dials Index Storage &Caching ForestDB: Brand new storage engine for high performance index caching and storage
  15. N1QL currently supports left-to-right JOINs, where the left-hand term produces the key of the right-hand term: SELECT * FROM beer JOIN brewery ON KEYS beer.brewery_id; Customers, including Apple and DirecTV, have requested additional JOINs where the right-hand term contains the key of the left-hand term: SELECT * FROM brewery JOIN beer ON KEY beer.brewery_id FOR brewery WHERE brewery.name = "21st Amendment Brewery Cafe";
  16. If a
  17. Parameters
  18. JOIN Covering index FOR clause
  19. Watson Roadmap: Array indexes New Shell Monitoring.