When RDF Alone Is Not Enough: Triples, Documents, and Data in Combination

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
When RDF Alone Is Not Enough
Stephen Buxton, MarkLogic
stephen.buxton@marklogic.com
Triples, Documents, and Data in Combination

© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 2
The Path
 Triples: What is Semantics?
 Documents and Data: What is MarkLogic?
 Triples and Documents, Triples and Data
 Some Use Cases
 Under the Covers (if there's time)

Data is stored in triples, expressed as: Subject : Predicate : Object
John Smith : livesIn : London
London : isIn : England
Query with SPARQL, gives us simple lookup .. and more!
Find people who live in (a place that's in) England
Semantics Is: A New Way to Organize Data
RDF
triples
John livesIn IsIn EnglandLondon

Triples Come in Different Formats
John livesIn London
<sem:triple>
<sem:subject> http://xmlns.com/foaf/0.1/name/"John"</sem:subject>
<sem:predicate> http://example.org/livesIn</sem:predicate>
<sem:object datatype="http://www.w3.org/2001/XMLSchema#string">"London"</sem:object>
</sem:triple>
{
"triple" : {
"subject": "http://xmlns.com/foaf/0.1/name" "John",
"predicate": "http://example.org/livesIn",
"object": { "value": "London", "datatype": "xs:string" }
}
<http://dbpedia.org/resource/John>
<http://dbpedia.org/ontology/LivesIn>
<http://dbpedia.org/resource/London> .
Turtle
JSON
XML
3 IRI’s
2 IRI’s,
1 string

Enter Semantics…
John livesIn IsIn EnglandLondon
Triples
Subject :Predicate :Object
Semantics is a simple and elegant way to model data as facts and relationships.
Semantics uses a data model called RDF that you query with SPARQL.

Semantics Is: A New Way to Organize Data
<http://example.org/dir/js> <http://xmlns.com/foaf/0.1/firstname> "John".
<http://example.org/dir/js> <http://xmlns.com/foaf/0.1/lastname> "Smith".
Example of RDF
SELECT ?person ?place
WHERE
{
?person <http://example.org/LivesIn> ?place .
?place <http://example.org/IsIn> "England" .
}
Example of SPARQL

Triple Store
Just a Triple Store is good when you want to …
 look up facts
– model atomic facts, relationships – reference data
 explore a graph
– model relationships/links
 combine sources
– triples are easy to share, easy to combine
 update some triples
– easy to insert/delete/update a single fact
– easy to insert/delete/update any part of the ontology (facts about the data)
 use the magic of inference
– simpler data modeling, data integration

<Institution><Author>
Biology
<category>
Neuro-Biology
<category>
London
<place>
New York
<place>
Works at
Located inLives inLived in
Studies
type of
Neurology
<category>Related To
Specializes In
<Institution>
Funded By
Specializes In
 Reference Data
 Metadata
 Provenance
 Modeling facts, relationships, links
Triple Store as Graph
More on Semantics

Hierarchical Era
―For your application data!"
 Application- and
hardware-specific
We Are The New Generation Database
Relational Era
“For all your structured data!”
 Normalized, tabular model
 Application-independent
query
 User control
Any Structure Era
“For all your data!”
 Schema-agnostic
 Massive scale
 Search and query
 Analytics
 Application services
 Faster time-to-results

Triples, Documents, Data: Architecture
STORAGE LAYER
Scalability
and
Elasticity
ACID
Transactions
Triple
Store
INTERFACE LAYER
mlcpJSON, XML,
RDF, Geo,
Binaries
REST
API
Graph /
SPARQL
QUERY LAYER
JS XQuery SPARQL
JavaScript XQuery SPARQLSQL
INDEXES / CACHE
Universal
Index
Geospatial
Index
Triple
Index
Triple
Cache
Automated
Failover
Reverse
Index

10.3
2.82
1.48 1.39
0.73 0.55
0.19 0.17 0.13 0.13
0
2
4
6
8
10
12
Triple Store Rankings (DB-engines, March 2015)

Where Triples Come From
 The World at Large
– Linked Open Data
– DBpedia
– GeoNames

– DBpedia
– GeoNames
 Facts from your domain
– Proprietary company data
– An industry-wide ontology such as FIBO
http://www.omg.org/spec/EDMC-FIBO/BE/1.0/Beta1/PDF/

– DBpedia
– GeoNames
 Facts from documents
– Document metadata (author, publish date, source, etc.)
– Entities and events in free-flowing text

– DBpedia
– GeoNames
 Facts from documents
– Document metadata
– Facts in free-flowing text
 Facts about data
– Data metadata
– Semantics of data
Data Metadata
 Provenance, source, security, bitemporal
The Semantics of Data
 Data Integration:
I added a new dataset where
old:customer <is the same as> new:ID
 Better queries:
I know the semantics of ownership
Acme just acquired Pinky, who owns Perky
so … Acme owns Perky
 Simpler, more accurate data modeling:
No need to (try to) represent all relationships explicitly

Why MarkLogic?
 MarkLogic is an Enterprise Triple Store
– Robust
– Horizontally scalable – billions of triples per box
– HA/DR features such as backup/restore, replication, automatic failover
– Government-grade security
 Triples + Documents + Data
– Complementary
– Intertwingled

Triples + Documents + Data: Complementary

Triples + Documents + Data: Intertwingled

Data Documents Triples
RDF
Enterprise Features
HA/DR, SECURITY, ACID TRANSACTIONS, SCALABILITY & ELASTICITY
JSON, XML
Flexible Data Model
Search & Query
BUILT-IN FULL-TEXT SEARCH

Benefits of a Triple Store
Just A Triple Store:
 Store and query hundreds of billions of
facts and relationships
 More context for your data
 Graph visualizations
 Reliance on a common standard
 Ability to infer new information

Benefits of a Document Store
Just A Document Store:
 Easily store heterogeneous data
(transactional data, records, free-text)
 Schema-agnostic for modeling freedom
and avoiding ETL
 Search flexibility and specificity
 Fast app development

Benefits of a Document Store and Triple Store Combined
All the benefits of each, plus:
 Docs can contain triples, Triples can
annotate docs, Graphs can contain docs
– Faster data integration using semantics as
the glue
– Ideal model for reference data, metadata,
provenance
– Ability to run really powerful queries
 Massive speed and scale
 Simplicity of a single unified platform
 Enterprise features (security, HA/DR, ACID
transactions,…)

Leading Organizations Using MarkLogic Semantics
 Intelligent Search
 Dynamic Semantic Publishing
 Semantic Metadata Hub
 Complex Data Integration
 Compliance
 Object-based Intelligence
Entertainment
Company
Agriculture
Company