Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Linking media, data,
and services
Ruben Verborgh
Ghent University – iMinds
The problem
The solution
The problems
The problem
The solution
The problems
How can semantic technologies
assist with multimedia analysis?
analysis

algorithms
Linked

Data
planning
knowledge

bases
reasoning
goal-driven
data-driven
• subrequest 1
…
• subrequest n Engine
EXIFJPEG
metadata
JPEG
inputrequest
The problem
The solution
The problems
multimedia
data
+
metadata
Supervisor
requested
metadata
description
Blackboard
requested
metadata
description
gathered
me...
multimedia
data
+
metadata
Supervisor
requested
metadata
description
Blackboard
requested
metadata
description
gathered
me...
query request
input
Collaborator
output
result
2.
3.
5.
1.
4.
Collaboratorinput output
Toolkit
Collaborator
Wrapper
input output
start
<Loft.jpg> a foaf:Image.
<Loft.jpg> foaf:depicts ?person.
input
goal
composition
{
?image a foaf:Image.
}
=>
{
?regionList a rdf:List.
_:region owl:oneOf ?regionList;
rdfs:subClassOf face:FaceRegion,
[ o...
{
?region a face:FaceRegion.
}
=>
{
?region imreg:regionDepicts ?face.
?face a face:Face;
face:isFaceOf ?person.
?person a...
{
?image imreg:region ?region.
?region imreg:regionDepicts ?face.
?face face:isFaceOf ?person.
}
=>
{
?image foaf:depicts ...
Example 7.8 Composition from image input to person request
8
>>>>>>><
>>>>>>>:
Iæ := (param1 = ) √ ( )
IFD := (regionList ...
execution
<Loft.jpg#xywh=45,121,51,51> a face:FaceRegion;
imreg:regionOf <Loft.jpg>.
<Loft.jpg#xywh=221,91,56,56> a face:FaceRegion;...
Example 7.8 Composition from image input to person request
8
>>>>>>><
>>>>>>>:
Iæ := (param1 = ) √ ( )
IFD := (regionList ...
_:face owl-s-sparql:bindsVariable "face";
owl-s-sparql:boundTo
[face:isFaceOf dbpedia:Koen_De_Bouw].
_:person owl-s-sparql...
dbpedia:Koen_De_Bouw ???
??? dbpedia:Bruno_Vanden_Broucke
failure
alternative
plan
CONSTRUCT WHERE {
dbpedia:Koen_De_Bouw ?p ?o.
}
CONSTRUCT WHERE {
?s ?p dbpedia:Koen_De_Bouw.
}
dbpedia:Koen_De_Bouw a dbpedia-owl:Actor;
rdfs:label "Koen De Bouw"@nl.
dbpedia:Bruno_Vanden_Broucke a dbpedia-owl:Actor;
...
{
?image foaf:depicts ?person
?person foaf:knows ?acquaintance.
}
=>
{
?image face:maybeDepicts ?acquaintance.
}.
knowledg...
recomposition
acquaintances to the parameter. Note that the actual process is slightly more
complex, but some details were omitted for b...
{
?image imreg:region ?regionA, ?regionB.
?regionA owl:differentFrom ?regionB.
?regionA imreg:regionDepicts [face:isFaceOf...
execution
dbpedia:Koen_De_Bouw
dbpedia:Bruno_Vanden_Broucke
dbpedia:Matthias_Schoenaerts
dbpedia:Bruno_Vanden_Broucke
dbpedia:Koen_D...
_:face owl-s-sparql:bindsVariable "face";
owl-s-sparql:boundTo [face:isFaceOf ?person].
_:person owl-s-sparql:bindsVariabl...
solution
dbpedia:Koen_De_Bouw dbpedia:Matthias_Schoenaerts
dbpedia:Koen_De_Graeve dbpedia:Bruno_Vanden_Broucke
The problem
The solution
The problems
ἀπὸ μηχανῆς θεός
deus ex machina
machina ex deo
reasoner
What are the research questions?
!
How can semantic technologies
assist with multimedia analysis?
What are the hypotheses?
!
This solution does analysis

better than… what?
How to evaluate this solution?
!
Is our solution

a good one or a bad one?
How can this solution fail?
!
Can (existing) knowledge

always fill the gap?
The problem
The solution
The problems
Can we link images like we link data?
Can we have a Linked Media Cloud?
Linking media, data,
and services
@RubenVerborgh

ruben.verborgh.org
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Próxima SlideShare
Cargando en…5
×

Linking media, data, and services

1.338 visualizaciones

Publicado el

Keynote at the 4th International Workshop on Linked Media (LIME) http://www.mico-project.eu/pages/lime-workshop-2016/

Publicado en: Internet
  • Sé el primero en comentar

Linking media, data, and services

  1. 1. Linking media, data, and services Ruben Verborgh Ghent University – iMinds
  2. 2. The problem The solution The problems
  3. 3. The problem The solution The problems
  4. 4. How can semantic technologies assist with multimedia analysis?
  5. 5. analysis
 algorithms Linked
 Data planning knowledge
 bases reasoning
  6. 6. goal-driven data-driven
  7. 7. • subrequest 1 … • subrequest n Engine EXIFJPEG metadata JPEG inputrequest
  8. 8. The problem The solution The problems
  9. 9. multimedia data + metadata Supervisor requested metadata description Blackboard requested metadata description gathered metadata Collaborator 3 semantic capability description semantic requirements description Collaborator 2 semantic capability description semantic requirements description Collaborator 1 semantic capability description semantic requirements description multimedia data + metadata r requested metadata description metadata Blackboard requested metadata description gathered metadata n 1 tic ity ion multimedia data + metadata requested metadata description metadata Blackboard requested metadata description gathered metadata
  10. 10. multimedia data + metadata Supervisor requested metadata description Blackboard requested metadata description gathered metadata Collaborator 3 semantic capability description semantic requirements description Collaborator 2 semantic capability description semantic requirements description Collaborator 1 semantic capability description semantic requirements description multimedia data + metadata r requested metadata description metadata Blackboard requested metadata description gathered metadata c y on r 1 ntic bility ption multimedia data + metadata visor requested metadata description metadata Blackboard requested metadata description gathered metadata or3 mantic ability cription tor2 semantic capability description rator1 semantic capability description
  11. 11. query request input Collaborator output result 2. 3. 5. 1. 4.
  12. 12. Collaboratorinput output Toolkit
  13. 13. Collaborator Wrapper input output
  14. 14. start
  15. 15. <Loft.jpg> a foaf:Image. <Loft.jpg> foaf:depicts ?person. input goal
  16. 16. composition
  17. 17. { ?image a foaf:Image. } => { ?regionList a rdf:List. _:region owl:oneOf ?regionList; rdfs:subClassOf face:FaceRegion, [ owl:hasValue ?image; owl:onProperty imreg:regionOf ]. } service rule
  18. 18. { ?region a face:FaceRegion. } => { ?region imreg:regionDepicts ?face. ?face a face:Face; face:isFaceOf ?person. ?person a foaf:Person. }. service rule
  19. 19. { ?image imreg:region ?region. ?region imreg:regionDepicts ?face. ?face face:isFaceOf ?person. } => { ?image foaf:depicts ?person. }. knowledge rule
  20. 20. Example 7.8 Composition from image input to person request 8 >>>>>>>< >>>>>>>: Iæ := (param1 = ) √ ( ) IFD := (regionList = ) √ (image = ) IFR := (f ace = , per son = ) √ (region = ) I≤ := ( ) √ (per son = ) Iæ and I≤ are the virtual start and end services, respectively. Iæ IFD IFR I≤ 62
  21. 21. execution
  22. 22. <Loft.jpg#xywh=45,121,51,51> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=221,91,56,56> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=535,118,43,43> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=734,83,69,69> a face:FaceRegion; imreg:regionOf <Loft.jpg>.
  23. 23. Example 7.8 Composition from image input to person request 8 >>>>>>>< >>>>>>>: Iæ := (param1 = ) √ ( ) IFD := (regionList = ) √ (image = ) IFR := (f ace = , per son = ) √ (region = ) I≤ := ( ) √ (per son = ) Iæ and I≤ are the virtual start and end services, respectively. Iæ IFD IFR I≤ 62
  24. 24. _:face owl-s-sparql:bindsVariable "face"; owl-s-sparql:boundTo [face:isFaceOf dbpedia:Koen_De_Bouw]. _:person owl-s-sparql:bindsVariable "person"; owl-s-sparql:boundTo dbpedia:Koen_De_Bouw. <Loft.jpg#xywh=221,91,56,56>
  25. 25. dbpedia:Koen_De_Bouw ??? ??? dbpedia:Bruno_Vanden_Broucke
  26. 26. failure
  27. 27. alternative plan
  28. 28. CONSTRUCT WHERE { dbpedia:Koen_De_Bouw ?p ?o. } CONSTRUCT WHERE { ?s ?p dbpedia:Koen_De_Bouw. }
  29. 29. dbpedia:Koen_De_Bouw a dbpedia-owl:Actor; rdfs:label "Koen De Bouw"@nl. dbpedia:Bruno_Vanden_Broucke a dbpedia-owl:Actor; rdfs:label "Bruno Vanden Broucke"@nl. ! dbpedia:Loft_%282008_film%29 dbpprop:director dbpedia:Erik_Van_Looy; dbpprop:name "Loft"@nl; dbpprop:producer dbpedia:Woestijnvis; dbpprop:starring dbpedia:Bruno_Vanden_Broucke, dbpedia:Filip_Peeters, dbpedia:Jan_Decleir, dbpedia:Koen_De_Bouw, dbpedia:Koen_De_Graeve, dbpedia:Matthias_Schoenaerts, dbpedia:Veerle_Baetens.
  30. 30. { ?image foaf:depicts ?person ?person foaf:knows ?acquaintance. } => { ?image face:maybeDepicts ?acquaintance. }. knowledge rule
  31. 31. recomposition
  32. 32. acquaintances to the parameter. Note that the actual process is slightly more complex, but some details were omitted for brevity. Example 7.14 Adapted composition, recovering from the face recognition failure 8 >>>>>>>< >>>>>>>: ... I0 æ := (param1 = ) √ ( ) I0 FR := (f ace = , per son = ) √ (region = , candidates = ) Iæ IFD IFR I≤ I0 æ I0 FR 67
  33. 33. { ?image imreg:region ?regionA, ?regionB. ?regionA owl:differentFrom ?regionB. ?regionA imreg:regionDepicts [face:isFaceOf ?personA]. ?regionB imreg:regionDepicts [face:isFaceOf ?personB]. } => { ?personA owl:differentFrom ?personB. }. knowledge rule
  34. 34. execution
  35. 35. dbpedia:Koen_De_Bouw dbpedia:Bruno_Vanden_Broucke dbpedia:Matthias_Schoenaerts dbpedia:Bruno_Vanden_Broucke dbpedia:Koen_De_Graeve
  36. 36. _:face owl-s-sparql:bindsVariable "face"; owl-s-sparql:boundTo [face:isFaceOf ?person]. _:person owl-s-sparql:bindsVariable "person"; owl-s-sparql:boundTo ?person. ({?person = dbpedia:Koen_De_Graeve.} {?person = dbpedia:Bruno_Vanden_Broucke.}) e:disjunction [a e:T]. <Loft.jpg#xywh=45,121,51,51>
  37. 37. solution
  38. 38. dbpedia:Koen_De_Bouw dbpedia:Matthias_Schoenaerts dbpedia:Koen_De_Graeve dbpedia:Bruno_Vanden_Broucke
  39. 39. The problem The solution The problems
  40. 40. ἀπὸ μηχανῆς θεός
  41. 41. deus ex machina
  42. 42. machina ex deo reasoner
  43. 43. What are the research questions? ! How can semantic technologies assist with multimedia analysis?
  44. 44. What are the hypotheses? ! This solution does analysis
 better than… what?
  45. 45. How to evaluate this solution? ! Is our solution
 a good one or a bad one?
  46. 46. How can this solution fail? ! Can (existing) knowledge
 always fill the gap?
  47. 47. The problem The solution The problems
  48. 48. Can we link images like we link data? Can we have a Linked Media Cloud?
  49. 49. Linking media, data, and services @RubenVerborgh
 ruben.verborgh.org

×