LOD2 Webinar Series: D2R and Sparqlify

Creating Knowledge out of Interlinked
Data

LOD2 W ebinar . 29.11.2011 . Page 1 http:/ l
/od2.eu

Data

LOD2 is a large-scale integrating project co-funded by the European
Commission within the FP7 Information and Communication Technologies
Work Programme. This 4-year project comprises leading Linked Open
Data technology researchers, companies, and service providers. Coming
from across 12 countries the partners are coordinated by the Agile
Knowledge Engineering and Semantic Web Research Group at the
University of Leipzig, Germany.

LOD2 will integrate and syndicate Linked Data with existing large-scale
applications. The project shows the benefits in the scenarios of Media and
Publishing, Corporate Data intranets and eGovernment.

/od2.eu

Data

Once per month the LOD2 webinar series offer a free webinar about
tools and services along the Linked Open Data Life Cycle.

Stay with us and learn more about acquisition, editing, composing,
connected applications – and finally publishing Linked Open Data.

/od2.eu

Data

Web-based Systems Group

• School of Business & Economics, Freie Universität Berlin
• Research focus: Linked Data technol
ogies for extending the Worl W ide W eb with a gl
d obal
data commons
• Funded Proj ects:
• LOD2 - Creating Knowl out of Interl
edge inked Data
• LATC - LOD Around The Cl ock
• Pl
anetData
• Visit us at: http:/ wbsg.de
/

/od2.eu

Data

Main Projects

• DBpedia is a community effort l by WBSG, AKSW and OpenLink Software to:
ead
• Extract structured information from W ikipedia
• Make this information avail e on the W eb under an open l
abl icense
• Interl the DBpedia dataset with other open datasets on the W eb
ink
• DBpedia Spotl Automatic annotation of free-text with DBpedia URIs
ight:

• Data Integration
• R2R: Transl W eb data that is represented using terms from different vocabul
ates aries into a singl target
e
vocabulary.
• Sil Toolfor generating RDF l between data items.
k: inks
• LDIF: Transl heterogeneous Linked Data from the W eb into a cl l target representation whil
ates ean, ocal e
keeping track of data provenance.

/od2.eu

Data

Outline

• D2R/ Sparql in the LOD2 Stack
ify
• The D2RQ Pl atform
• The D2RQ Mapping Language
• Exampl and Demo
e
• Avail ity
abil
• Sparql (Cl Stadl
ify aus er)
• Q &A

/od2.eu

Data

D2R/Sparqlify in the LOD2 Stack

/od2.eu

Data

The D2RQ Platform

• System for accessing rel
ational databases as virtual RDF graphs
• Offers RDF-based access to the content of rel
ational databases without having to repl
icate it
into an RDF store

• Features:
• query a non-RDF database using SPARQL
• access the content of the database as Linked Data over the W eb
• create custom dumps of the database in RDF
• access information using the Apache Jena API

/od2.eu

Data

Components

• The D2RQ Pl atform consists of:
• D 2 R Q M a p p i n g L a n g u a g e , a decl arative mapping l
anguage for
describing the relation between an ontol and an rel
ogy ational data model
.
• D 2 R Q E n g i n e , uses the mappings to rewrite SQL queries against the database
and passes query resul up to the higher l
ts ayers of the frameworks
• D 2 R S e r v e r , an HTTP server that provides a Linked Data view, a HTML view
for debugging and a SPARQL Protocol endpoint over the database.

/od2.eu

Data

Architecture

/od2.eu

Data

D2RQ Mapping Language

• Decl
arative language for mapping rel
ational database schemas to RDF vocabul
aries and
OWL ontol ogies.
• N3 based syntax
• Very fl e
exibl
• Usual workfl auto-generate mapping from DB schema, then customize
ow:

/od2.eu

Data

Mapping process

/od2.eu

Data

Example

• Existing database which stores information about:
• Conferences
• Papers
• Authors
• Topics
• W e want publ this database as RDF
ish
• W e wil use the International Semantic W eb Community (ISW C) Ontol
l ogy.

/od2.eu

Data

Define DB connection

map:MyDatabase a d2rq:Database;
d2rq:jdbcDSN "jdbc:mysql://localhost/mydb";
d2rq:jdbcDriver "com.mysql.jdbc.Driver";
d2rq:username "user";
d2rq:password "password".

• d 2 r q : D a t a b a s e defines a JDBC connection to a l or remote rel
ocal ational
database
• d 2 r q : j d b c D S N specifies the JDBC database URL
• Typicaly of the form: jdbc:subprotocol:subname
l
• d 2 r q : j d b c D r i v e r specifies the JDBC driver for the database

/od2.eu

Data

Define your entities

map:People a d2rq:ClassMap;
d2rq:uriPattern “http://.../people/@@User.ID@@”.

(SQL fragments in red)

• d 2 r q : C l a s s M a p represents a cl or a group of simil cl
ass ar asses
• A cl map defines how instances of the cl are identified
ass ass
• d 2 r q : u r i P a t t e r n specifies a URI pattern that wil be used to identify
l
instances of this cl map.
ass

/od2.eu

Data

Define your entities

d2rq:uriPattern “http://.../people/@@User.ID@@”;
d2rq:condition “User.deleted=0”.

(SQL fragments in red)

• d 2 r q : c o n d i t i o n specifies an SQL W HERE condition
• An instance of this cl wil onl be generated for database rows that satisfy the condition
ass l y
• Conditions can be used to hide parts of the database from D2RQ

/od2.eu

Data

Add properties to entities

d2rq:uriPattern “http://.../people/@@User.ID@@”;
d2rq:condition “User.deleted=0”;
d2rq:class foaf:Person .

(SQL fragments in red, RDFS/OWL vocabulary in blue)

• d 2 r q : c l a s s rel the generated entity to a OWL/
ates RDFS cl
ass
• W e use the P e r s o n cl from the FOAF vocabul
ass ary

/od2.eu

Data


map:name a d2rq:PropertyBridge;
d2rq:belongsToClassMap map:People;
d2rq:property foaf:nick;
d2rq:column “User.name”.

map:mbox a d2rq:PropertyBridge;
d2rq:property foaf:mbox;
d2rq:uriPattern “mailto:@@User.email@@”.

• A d 2 r q : P r o p e r t y B r i d g e rel a database col to an RDF property.
ates umn
• Here we use properties from the FOAF vocabul as wel
ary l

/od2.eu

Data


map:mbox_sha1 a d2rq:PropertyBridge;
d2rq:property foaf:mbox_sha1sum;
d2rq:sqlExpression
“SHA1(CONCAT(‘mailto:’, User.email))”.

• d 2 r q : s q l E x p r e s s i o n generates l iteral val by eval
ues uating a SQL
expression.
• Note that querying for such a computed val might put a heavy l on the database.
ue oad
• W e compute the SHA1 sum from the user email address

/od2.eu

Data

Link your entities

map:Photos a d2rq:ClassMap;
d2rq:uriPattern “http://.../photo/@@Photo.ID@@”;
d2rq:class foaf:Image .

• W e define a second cl mapping for photos
ass
• In the next step, we wil interl person with their photos
l ink

/od2.eu

Data

Link your entities

map:photo a d2rq:PropertyBridge;
d2rq:property foaf:made;
d2rq:uriPattern “http://.../photo/@@Photo.UserID@@”.


• W e can use the al ready presented syntax to interl persons to their photo
ink
• Photo.UserID is a foreign key to User.ID

/od2.eu

Data

Link your entities
• Better, with l repetition:
ess

map:photo a d2rq:PropertyBridge;
d2rq:property foaf:made;
d2rq:join “User.ID = Photo.UserID”;
d2rq:refersToClassMap map:Photos .

/od2.eu

Data

Mapping Overview

/od2.eu

Data

• Demo

/od2.eu

Data

Availability

• D2RQ can be downl
oaded from the official homepage at:

• http:/ d2rq.org/
/

• Support is provided through the official mail l
ing ist:

• d2rq-map-devel l
@ ists.sourceforge.net

• The l
atest source code is avail e from the proj s Git repository:
abl ect'

• https:/ github.com/
/ d2rq/d2rq

• D2RQ is l
icensed under the terms of the Apache Software Licence
/od2.eu

Data

Developers

/od2.eu

Data

Database Compatibility

• Supported databases
• Oracl e
• MySQL
• PostgreSQL
• SQL Server
• HSQLDB
• Interbase/Firebird
• ODBC data sources
• Works with some l
imitations.
• Other databases
• May or may not work. By defaul D2RQ interacts with the database using the SQL-92 standard.
t,
Any compatibl database should work out of the box. W e are interested in reports about D2RQ
e
on other databases.

/od2.eu

Data

Current Work

• D2RQ is activel devel
y oped
• Work on supporting RDB2RDF (Direct Mapping und R2RML) in the next 6 weeks

/od2.eu

Data

Sparqlif
y
Project Page: http://aksw.org/projects/Sparqlify
Source Code: https://github.com/AKSW/Sparqlify
/od2.eu

Data

About me

• Cl Stadl
aus er
• Austria
• PhD Student at the University of Leipzig since 2011
– In the Agil Knowl Engineering and Semantic W eb (AKSW ) research group, headed by
e edge
Soeren Auer.
• Research Interests: SpatialData Management, SPARQL-SQL query rewriting
and optimization, Data integration.

/od2.eu

Data

Agile Knowledge Engineering and
Semantic Web Research Group

• Founded in 2006
• 25+ Researchers
• 3 Sub groups

• Goal
s
– Contributing to the advancement of science in Semantic W eb, Knowl Engineering, Software Engineering
edge
– Cost efficient, high-impact R&D, which proves usefulness at an earl stage
y
– Bridge the gap between research resul and appl
ts ications

• Committed to Open Source, Open Access, and Open Knowl movements
edge

/od2.eu

Data


• EU Funded Proj
ects:

– Linked Open Data 2 (LOD2)

– LOD Around the Cl (LATC)
ock

– Open Data Portal (ODP)

– Semantic Content Management Systems for Enterprise Knowl Management and News Mining (SCMS)
edge

– OntoW iki - Semantic Colaboration for Knowl Management, E-Learning and E-Tourism
l edge

/od2.eu

Data


• Further Proj
ects
– SlideW iki
• Sl ideW iki is a colaboration pl
l atform which enabl communities to buil share and pl onl presentations.
es d, ay ine
– LinkedGeoData
• Making OpenStreetMap data avail e in the Semantic W eb
abl
• Motivation for Sparql ify
– LIMES
• Very fast tool for interl inking RDF knowl bases.
edge
– DBpedia Live
• Synchronization of DBpedia with W ikipedia
– …

• Find more at
– http:/ aksw.org/ ects
/ Proj

/od2.eu

Data

Structure

• Introduction
• View Definition Exampl
e
– based on chalenges encountered with LinkedGeoData
l
• Launching Sparql Server
ify
• Demonstration
• Initial Results of the Performance
Evaluation
• Concl
usion & Future Work
• Outro

/od2.eu

Data

Introduction

• S p a r q l i f y is a SPARQL-SQL rewriter that enabl one to define RDF views on rel
es ational
databases and query them with SPARQL. Currentl onl PostgreSQL is supported.
y y

• Inputs
– PostgreSQL Database, Set of View Definitions, Sparql Query

• Features
– Intuitive View Definition Syntax
– SPARQL queries are rewritten into a singl SQL query
e
• Give as much control as possibl to the query optimizer of the underl RDBMS
e ying
– High expressivity
• Language and Data type Tags can originate from col umns
• Constraints can be stated for tuning the rewriting process
– Initial support for geospatial predicates
• Can be extended to enabl the use of arbitrary SQL predicates on the SPARQL l
e evel

/od2.eu

Data

View Definition Example: Mapping
the table “points_of_interest”

id type geom
1 lgdo:Bakery (1, 1)
2 lgdo:School (2, 2)
3 lgdo:Pub (3, 3)

On the following slides, Prefix Declarations are omitted for brevity

/od2.eu

Data


id class geom Create View pois As
Construct { …
2 lgdo:School (2, 2)
3 lgdo:Pub (3, 3)

/od2.eu

Data


id type geom Create View pois As
Construct {
?s a ?t .
2 lgdo:School (2, 2) ?s geom:geometry ?geo .
3 lgdo:Pub (3, 3) }
With …

/od2.eu

Data


Construct {
?s a ?t .
3 lgdo:Pub (3, 3) }
With
?s = spy:uri(concat(“http://ex.org/”, ?id))
….

/od2.eu

Data


Construct {
?s a ?t .
3 lgdo:Pub (3, 3) }
With
?s = spy:uri(concat(“http://...”, ?id))
?t = spy:uri(?type)
?geom = spy:typedLiteral(?geom, ogc:WKTLiteral)
From
…

/od2.eu

Data


Construct {
?s a ?t .
3 lgdo:Pub (3, 3) }
With
?t = spy:uri(?type)
From
points_of_interest;

/od2.eu

Data


Construct {
?s a ?t .
3 lgdo:Pub (3, 3) }
With
?t = spy:uri(?type)
Constrain
?t prefix “http://linkedgeodata.org/ontology/”
From
points_of_interest;

/od2.eu

Data

the table “resource_label”

resource label language
lgdo:Bakery Baeckerei de
lgdo:Bakery Bakery en
lgdo:School Schule de

/od2.eu

Data

the table “resource_label”

resource label language
lgdo:Bakery Baeckerei de
lgdo:Bakery Bakery en
lgdo:School Schule de
Create View labels As
Construct {
?s rdfs:label ?l .
}
With
?s = spy:uri(?resource)
?l = spy:plainLiteral(?label, ?language)
Constrain
?s prefix “http://linkedgeodata.org/ontology/”
From
resource_labels;
/od2.eu

Data

View Definition Example: Adding a
set of static triples

Create View static_triples As
Construct {
lgdo:Bakery a owl:Class .
lgdo:School a owl:Class .
lgdo:Pub a owl:class
};

/od2.eu

Data

View Definition File Syntax

Prefix Declarations

Create View {name} As
Construct {
{triple patterns}
}
With
{variable bindings}
Constrain
{constraint expressions}
From
logical table (table, view or SQL query);

… More View Definitions …

/od2.eu

Data

View Definition Example:
Create View
Wortschatz
view_co_n As
Construct {
?a wso:coOccursDirectlyWith ?b .
?x owl:annotatedSource ?a .
?x owl:annotatedProperty wso:coOccursDirectlyWith .
?x owl:annotatedTarget ?b .
?x wso:frequency ?f .
?x wso:sigma ?s .
}
With
?a = spy:uri(concat('http://aksw.org/wortschatz/word/', ?w1_id))
?b = spy:uri(concat('http://aksw.org/wortschatz/word/', ?w2_id))
?x = spy:uri(concat('http://aksw.org/wortschatz/co-occurence/direct/',
?w1_id, '/', ?w2_id))
?f = spy:typedLiteral(?freq, xsd:long)
?s = spy:typedLiteral(?sig, xsd:long)
From
[[SELECT w1_id, w2_id, freq::bigint, sig::bigint FROM co_n]];

Escape SQL queries in double brackets

/od2.eu

Data

Launching Sparqlify

• Downl from git, buil with
oad d
– mvn assebl
y:assembl
y
• Run
– j -cp target/
ava sparql
ify-0.0.1-SNAPSHOT-j
ar-with-dependencies.j RunEndpoint [options]
ar
• Options are
– Server Configuration
• -c Config fil containing the mapping definitions
e
• -P Server port [defaul 7
t 531]
– Database settings
• -h Hostname of the database (e.g. l host or l host:5432)
ocal ocal
• -d Database name
• -u User name
• -p Password
– Qual of Service
ity
• -n Maximum resul set size
t
• -t Maximum query execution time (excl uding rewriting time)

/od2.eu

Data

Demonstration

/od2.eu

Data

Initial Results of the
Performance Evaluation

• Initialperformance comparision on BSBM 1 mio dataset on PostgreSQL:
– (Times per Query Mix)
– D2R Fast Mode Disabl ~8sec
ed:
– D2R Fast Mode Enabl ~3sec
ed:
– Sparql 4 sec
ify:
– Performance is comparabl to D2R.
e

• Mixed resul for the LinkedGeoData schema:
ts
– Simpl queries work wel on the LGD schema
e l
– Compl queries are troubl
ex esome (timeouts) on a compl OSM dump as the PostgreSQL optimizer makes suboptimal choices.
ete

/od2.eu

Data

Conclusion and Future Work

• Sparql provides an intuitive Mapping Syntax
ify

• Originaly devel
l oped for the LinkedGeoData use-case
– Spatial predicate support, arbitrary predicate support pl
anned.
– URIs, l
anguage and datatype tags can be mapped from col umns of the DB.
– Queries are rewritten into a singl SQL statement, in order to give as much control to the query optimizer of the underl
e ying
DBMS as possibl e.

• Initialperformance resul seem to be comparabl to D2R
ts e
– More extensive testing has yet to be done

• Bugfixing

• Additionalfeatures
– Especialy support for the COUNT keyword
l

/od2.eu

Data

Contact

• Proj Page
ect
– http:/ aksw.org/ ects/
/ proj Sparql
ify

• Source Code
– https:/ github.com/
/ AKSW /Sparql
ify

• AKSW Research Group
– http:/ aksw.org
/

• My Work Page
– http:/ bis.informatik.uni-l
/ eipzig.de/ ausStadl
Cl er

• My Email
– cstadl informatik.uni-l
er@ eipzig.de

/od2.eu

Data

Thank you for your attention!

Q &A

/od2.eu

Data

Credits

Jingle R.E.M., Martin Kaltenböck, Florian Kondert
Coordination Thomas Thurner
Martin Kaltenböck
Moderation Martin Kaltenböck
Presented by Robert Isele & Claus Stadler

/od2.eu

Data

Hope you enjoyed staying with us – if you need more detailed
information, visit us at www.lod2.eu and let us know how we can
improve to meet your expectations!

Don’t forget to register for our next webinar

22.05. 2012 – Cloud View (Exalead Dassault Systems, France)
19.06. 2012 – PoolParty Thesaurus Manager (SWC, Austria)
Have a great day and don’t forget ...

http:/ l
/od2.eu
/od2.eu

Data

http:/ l
/od2.eu
/od2.eu

Data

Why another SPARQL – SQL
Rewriter?

• There is
– Virtuoso RDF Views
– D2R
– Revel Spyder
ytix
– Asio Semantic W eb Bridge for Relational Databases
– ODE Mapster, RDBToOnto
– Soon further impl
ementations of R2RML
– Ultrawrap
– …

/od2.eu

Data

Motivation

• Map OpenStreetMap data to RDF
– Taken approach
• Downl a OSM pl fil (> 10GB compressed), pipe each OSM entity (node, way, rel
oad anet e ation) through a custom Java
RDF mapper, and l the data into Virtuoso
oad
• Impl emented a LiveSync on top of that
• Repeat the dump process after each change in the mappings
• Takes more than 2 days.
– Goal
• Immediate effect of a change in the mappings
• Reuse of Osmosis' LiveSync
– Possibl Sol
e ution
• Keep the mapping information in the rel ationaldatabase, and use a RDB-RDF mapper for querying it.
– However: Back in April 2011, none of the existing RDB-RDF sol utions seemed suitable
• Lack of support for spatial predicates
• Eval uations of Sparql ters in memory
-Fil
• No support for creating l s where the l
iteral anguage tag or datatype are stored in the database.

/od2.eu

Data

Motivation

• LinkedGeoData proj Convert OpenStreetMap (OSM) data as RDF
ect:
– (http:/ l
/inkedgeodata.org)

• Main tabl of the OSM Schema (Excerpt):
es
– Nodes(id, geom, tstamp)
– NodeTags(node_id, k, v)
(place, city)
(name, Leipzig)
– Ways(id, geom, tstamp)
– WayTags(way_id, k, v)

– WayNodes(way_id, sequence_id, node_id)

/od2.eu

Data

Challenges with OpenStreetMap
data
• Geometry datatype
• URIs and l
anguage tags stored in database tabl
es

Nodes (OSM)
node_id k v
1 amenity school

Additional mappings tables for LGD

k v property object
lgd_map_resource_kv
amenity school rdf:type lgdo:school

k v label language
lgd_map_resource_labels
amenity school Schule de
Labels imported from TranslateWiki
/od2.eu

Data

Rewriting process

/od2.eu

Data

Rewriting process

• Rewriting process
– View Candidate Finding
• Given a SPARQL query, find an appropriate subset of the views for answering the query
– Rewriting
• After the candidates have been identified, transl the SPARQL al
ate gebra to SQL algebra.
• Thereby do book-keeping of how the SPARQL variabl are reconstructed from the SQL col
es umns.
– Resul Set Rendering
t
• Execute the SQL query, construct the RDF according to the SPARQL variabl bindings, serial the resul
e ize t.

/od2.eu

Data

View Candidate Finding

• Based on
Le, Wangchao and Duan, Songyun and Kementsietsidis, Anastasios and Li, Feifei and Wang, Min
R e w r it in g Q u e r ie s o n S P A R Q L V ie w s ,
In WWW 2011

/od2.eu

LOD2 Webinar Series: D2R and Sparqlify

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a LOD2 Webinar Series: D2R and Sparqlify

Similar a LOD2 Webinar Series: D2R and Sparqlify (20)

Más de LOD2 Creating Knowledge out of Interlinked Data

Más de LOD2 Creating Knowledge out of Interlinked Data (17)

Último

Último (20)

LOD2 Webinar Series: D2R and Sparqlify