Más contenido relacionado La actualidad más candente (20) Similar a Enabling access to Linked Media with SPARQL-MM (20) Enabling access to Linked Media with SPARQL-MM1. Enabling access to Linked Media with
Thomas Kurz, Kai Schlegel, Prof. Dr. Harald Kosch
3rd International Workshop on Linked Media,WWW 2015, Florence
2. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
The MICO Project and me
• www.mico-project.eu
• 36 Months Project
• started 10/13
• 3.4 Million Euro Project
• 7 Partners
3. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
The MICO project idea
4. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
The MICO project idea
6. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Multimedia Retrieval
on theWeb of Data is finding
(fragments of) resources of an
structured and unstructured
nature (text, image, video,
concepts etc.) that satisfy an
information need.
Extension of the standard definition of information retrieval in:
Christopher D. Manning and Prabhakar Raghavan.An Introduction to Information Retrieval. Cambridge University Press, 2009.
7. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Multimedia Query Languages
1. SQL-like approaches (WebSSQL or SQL/MM)
2. Object oriented approaches (MOQL or POQLMM)
3. Focusing XML metadata (MMDOC-QL or XQuery)
4. Visual Query Languages, like (MQueryVisual-MOQL)
5. Query-By-Example approaches, like WS-QBE
6. Metadata agnostic like MPQF
A Survey with ~40 MM Query Languages is coming soon …
8. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Multimedia Query Languages:
Requirements and Features
• Spatial Operation
• Temporal Operation
• Freetext Search
• Functions: Aggreg., Sort., Group., Set-Fnct.
• Similarity Search (Top-k)
• Event / Object Concept
• Presentation Part
General Requirements:
Application Independency, Formal Semantics, Extendability, Closed, etc.
Specific Features:
• Fuzzy terms
• Relevance Feedback
• Camera Motion
• Object Motion (Trajectory)
• Query by Example (Image, Video,
Sketch, Humming)
• Condition Weighting
• Low-Level-Features (Audio, Video)
9. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Semantic Web Query Languages
• There has been many in the past, but SPARQL
(SPARQL Query Language for RDF) is the winner
of the "natural selection".
• With SPARQL 1.1 many missing features have
become part of the standard.
• SPARQL is pretty good to extend (even without
changing the basic syntax)
BUT: It's missing some important features
10. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
What we want to reach
"Find scenes where Barack Obama is left beside the
Greenpeace Managing Director during an UN
climate conference ordered by length."
Features needed:
• Object Identification
• Temporal Operation
• Spatial Operation
• Freetext Search
• Sorting by Temporal Selector
11. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
How could it look like?
"Find scenes where Barack Obama is left beside the Greenpeace MD during the
UN climate change summit ordered by length."
SELECT ?scene WHERE {
?a3 oa:hasBody ?event; oa:hasTarget ?s3. # there are resources about an event
?event schema:Event; schema:summary ?description.# which has a description
FILTER mm:fulltext-search(str (?description), "UN climate conference","en")
# about 'UN climate conference'
?a2 oa:hasBody :Barack_Obama; oa:hasTarget ?s2. # and there are resources about Obama
?a1 oa:hasBody ?p1; oa:hasTarget ?s1. # and there are resources
?p1 :ceo_of dbpedia:Greenpeace. # about the MD of Greenpeace.
FILTER mm:leftBeside(?s2,?s1) # Obama has to be at the left of the MD and
FILTER mm:intersects(?s3,?s2) # has to be appear at the same time like the event.
BIND (mm:boundingBox(?s1,?s2) AS ?scene) # Wrap the results to scenes
ORDER BY DESC(mm:duration(?scene)) # and show the longest first.
12. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Goals
• Bring Multimedia Functionalities to SPARQL
(especially spatio-temporal support)
• Give transparent access to common
Metadata Models (Web Annotation
Selectors, Media Fragment URIs)
• Provide efficient evaluation
• Make it more easy for users
13. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Spatio-Temporal Object Model
14. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Spatio-Temporal Object
Model: Example: Rectangle
15. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Supported Standards
• Media Fragment URIs
• Spatial Fragment (currently Rectangle)
• Temporal Fragment (currently Interval)
• Web Annotation Model
• Media Fragment Selectors (http://www.w3.org/TR/media-frags/)
• SVG Basic Shapes (http://www.w3.org/TR/SVG/shapes.html)
http://example.org/video.mp4#t=1,5&xywh=10,20,30,30
16. © Salzburg Research Forschungsgesellschaft 2012. No reproduction without written permission. Certified in accordance with ISO 9001:2008
SPARQL-MM Functions
Spatial Topological Relations
Spatial Predicates
based in the
Dimensionally Extended
nine-Intersection Model
+(DE-9IM)
17. © Salzburg Research Forschungsgesellschaft 2012. No reproduction without written permission. Certified in accordance with ISO 9001:2008
SPARQL-MM Functions
Spatial Directional Relations
18. © Salzburg Research Forschungsgesellschaft 2012. No reproduction without written permission. Certified in accordance with ISO 9001:2008
SPARQL-MM Functions:Temporal Relations
19. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Multimedia Tests and
Accessors (not complete yet)
xsd:boolean mf:isMediaFragmentURI(Resource r)
xsd:boolean mf:isMediaFragmentSelector(Resource r)
xsd:boolean mf:isSVGSelector(Resource r)
mo:Rectangle mf:getBoundingBox(mo:SpatialEntity e)
xsd:double mf:getArea(mo:SpatialEntity e)
mo:Point mf:getXY(mo:Rectangle r)
xsd:double mf:getWidth(mo:Rectangle r)
xsd:double mf:getWidth(mo:Rectangle r)
xsd:double mf:getDuration(mo:TemporalEntity e)
xsd:double mf:getStart(mo:TemporalEntity e)
xsd:double mf:getEnd(mo:TemporalEntity e)
20. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Fulltext Search
Based in Apache Marmotta Fulltext Search*
• mf:fulltext-search(text, query, [language]):
searches “text” for the words occurring in “query”, optionally
applying the language-specific processing for the given language;
query is a simple text literal (list of words) without any boolean
connectors; words are AND connected, i.e. all words have to be
found in the text for a successful match.
• mf:fulltext-query(text, query, [language]):
searches “text” using the boolean query string passed in “query”,
optionally applying language-specific processing for the given
language; query is a boolean query string following the syntax
used by PostgreSQL.
* http://marmotta.apache.org/kiwi/sparql.html
21. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Status of Implementation
• SPARQL-MM 1.0 is out for a while, but:
• it is slow (based on Sesame Function Extensions)
• it is reduced
• it is very prototype (therefore still Snapshot)
https://oss.sonatype.org/content/repositories/snapshots/com/github/tkurz/sparql-mm/
• SPARQL-MM 2.0 will be available soon (planned for 8/2015)
• it is much faster (based on SQL-MM using PostGIS)
• it will be available as Open Source Marmotta Module*
• is has a big feature set
* maybe we should create a LIME Module that include implementations for various Linked Media
related things like Web Annotation endpoints etc. ?
22. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Short Demo
<repositories>
<repository>
<id>oss-sonatype</id>
<name>oss-sonatype</name>
<url>
https://oss.sonatype.org/content/repositories/snapshots/
</url>
</repository>
</repositories>
<dependency>
<groupId>com.github.tkurz</groupId>
<artifactId>sparql-mm</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
23. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Future Plans
• SPARQL like for shapes
mm:like(mo:Circle(20,20,100), ?f)
• Extend Model to Layers
mm:behind(?f1, ?f2)
• SPARQL like for Resources
mm:like(?u1, ?u2, 0.4)
24. © Salzburg Research Forschungsgesellschaft 2015. No reproduction without written permission. Certified in accordance with ISO 9001:2008
Thanks for your Attention!
Thomas Kurz
Knowledge and Media Technologies
Salzburg Research Forschungsgesellschaft m.b.H.
Jakob-Haringer-Straße 5/III | Salzburg,Austria
Tel. +43 662 2288-253| Fax +43 662 2288-222
thomas.kurz@salzburgresearch.at