A Lightweight Ontology for Linking Images

A Lightweight Ontology for
Linking Images

Pat Hayes, Margaret Warren
© 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

what we are/and are not doing
• images themselves are our main topic, rather than as
illustrations of other topics, linking information to
and from them and linking the images themselves
• Interested in using linked data to represent the kinds
of things artists themselves say about their work (or
the person doing the markup)
• What we don’t do: expect people to agree on artistic
interpretations or emotional content of images
• ignore our paper in the proceedings – it was a
working draft and we have made significant changes.


foaf:depiction


Snapshot, line drawings, news photos, paintings, fine art

Image identity
• By Image, we mean a digital image, typically
one that can have associated EXIF data.
• So, other kinds of "images" (drawings, paintings,
non-digital photographs, stained glass, etc..)
have to be classified as Works.
• A work may be a physical thing with dimensions,
a location, a time and place of creation, etc. that
is separate from those same properties in an
image; a digital image can be a work.
• Basic relation between Work and Image is
facsimile, a kind of 'visually same As' relation.


Broader notion of depicts

All images depict the person: McKenzie Oerting

Depiction can include imaginary, biblical, mythological notions


What is the subject?


shows, depicts, subject, conveys

Shows: trees, grass, house,
window, ventilator
Shows: head
Depicts: particular type of car,
Depicts: hands
each of the people
Conveys: grief, sadness
Subject: group looking at a car


Do all images depict something?

Depict diamonds? Shapes? Or Nothing…

Pictorial elements
Basic graphical 'unit' of depiction is a pictorial element.
This includes 'boxes' but also lines, regions, areas,
patches of color, texture, etc.. “

elements of art - The basic components used by the artist
when producing works of art. Those elements are color,
value, line, shape, form, texture, and space. The elements
of art are among the literal qualities found in any artwork.
(Artlex, 1996-2010)

Pictorial elements are related by inclusion: 3/23/2010
© 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions

Depicts vs. looksLike
Pictorial Elements:
Shape of thing that
looks like a
female torso
Shape of thing that
looks like a
representation of
a brain

area that looks like
the torso is
sitting astride a seat of
some sort
Title: Surfing With Psyche

Depicts: egg, a test tube, a photographic transparent layer, a piece of jewelry

Pictorial Elements can be 'empty'

A reclining nude is depicted by
white space defined by an outline.
The white space is a pictorial
element.

Cataloging pictorial elements is a
large task. As a beginning, we
distinguish visible/implied and
line/area. Here, the nude is
depicted by an implied area. All
the visible elements are red lines.


backgrounds

hasPictorialBackground hasDepictedBackground
all white, not necessarily sky, clouds, not necessary
able to define with a region to define with a region


Angels in the clouds
depicts:
Blue Angels ,jets,
planes, formation

conveys:
military power?

subject: specific formation?

hasDepictedBackground:
sky, clouds
depicts: angel

shows: clouds

region contains a shape
(PictorialElement) that
looksLike an Angel
(iconic)

depicts: clouds


The ontology so far


future work


http://www.purl.org/net/mw

http://www.carmapro.com/mw/mw.rdf


Extra slides follow


Looking like an archetype
• Some shapes are best described by what they look like iconically.

• If we can use RDFS or OWL Full, this could be described as 'looking like' a
class; but this will break OWL-DL reasoners.
• Or, we could treat a SKOS concept as an archetype, but this is not (yet) a
widely accepted convention.

visiblePictorialElements
looksLike iconic heart
hasColor Red
looksLike iconic dancing figures
hasColor yellow orange red

hasPictorialBackground: white


What's in an image?
• We conventionally say that depicted things
are "in" an image.
• Is this a philosophical mistake? Maybe. But
rather than try to correct it, the ontology tries
to avoid it.
• Basic methodological rule for linked data:
don't try to repair "bad" intuitions.


shows: person, depicts: face in hands shows: broken headlight, turn signal
subject: crying conveys: grief or despair, etc. depicts: front fender of school bus
subject:
conveys: beauty in decay, decline of
education

Can any Thing be depicted? Do we
need a class of 'depictable things'?
Things depicted include
(keywords):

• the car (specific type)

• people (specific people)

• a group of 7 people

• a building
Subject: group studying a car?
• trees
one of the people?
the car itself? • brown grass


Adult, Adults Only, Agriculture, Beautiful People, Beauty, Beauty In Nature,
Biting, Branch, Color Image, Concepts, Custard Apple, Desire, Enjoyment,
Food, Freshness, Fruit, Green, Gripping, Growth, Harvesting, Healthy Eating,
Healthy Lifestyle, Horizontal, Human Face, Human Hand, Human Mouth,
Hungry, Leaf, Lifestyles, Meal, Nutrient, One Person, One Woman Only, One
Young Woman Only, Only Women, Open, Opening, Organic, People,
Photography, Picking, Plant, Southeast Asian Ethnicity, The Human Body,
Tree, Vegetarian Food, Women, Young Adult


What can be depicted?
• FOAF: People.
• DBPedia: People, places, events,…
• Umbel (Cyc): VisibleThings
• Visual Resources Association: myths, legends, intentions, …
• We provide a different property conveys for the relationship
between an image and "un-depictable" things such as mood,
idea, intention, emotion, etc..


http://www.gettyimages.com/Search/Search.aspx?query=z.i.H4
sIAAAAAAAEAO29B2AcSZYlJi9tynt_SvVK1-
B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmVdZhZAz
O2dvPfee--
999577733ujudTif33_8_XGZkAWz2zkrayZ4hgKrIHz9-
fB8_In7dfLn91etf49f4NX6PX_dskV3kvyb9mtD_f9N5ns3S9fKqz
larYnnxa_7f9PwaG59f0_xM6qpqX2Z1tmjMZ78W_r-
78xvSj1_v3evr5vSdeePX9P7mltm5Bai_4-
VfOzuf2g78P35tfLDD8Bete3PR_rr6-2_xa3DX9tc99-s99-t99-s-
__rrUB-lhWf_Qne_Lv1RLyw24Z-_Pj7axV-_Hn8-
sTA6f_Nr5yEU-
ydD2bFQzjtQ3N_8Wm1e6_zJUPYslLoDxf1NBM0tDPMH_s_U
3QV1iQL5yr4f_PVr55PGfmP-
4BnJXY_e779W7pp7v__azWplP8cfBqPfyr54OXUvXk5_bf39t8
TfO_a3ffvbLn77dara4zb7F381yf2vzF94fu3Z1dL8_muaP8BSv0
6z9kYb_PVr55nDz_-Dfl_5X9g_ft0mgBb--etM83futeCvX_vds-
dOGugP-
0VWe2h7f_xGzbRa5U_Wy1npUT_41AD8ccB5sHMfv_86xGpr2
zz46_8Bs9pv0zoEAAA.


Linked Image Data
• Linked data…
• Newest manifestation of the semantic web:
use RDF to create a web of semantic links
between data items.
• Meanings of links are described by lightweight
(small, intuitive, useful) ontologies.


Linked data, not data silos


Linked data is link + description
• RDF descriptions use URIs as names to refer to things; linking means using
those URIs to also connect to the described thing, or a source of more
information about it.
• This can be smoothly integrated into HTML using RDFa.
<foaf:knows> <foaf:Person rdf:about="http://danbri.org/foaf.rdf#danbri">
<foaf:mbox_sha1sum>362ce75324396f0aa2d3e5f1246f40bf3bb44401</foaf:mbox_sha1sum>
<foaf:name>Dan Brickley</foaf:name>
<rdfs:seeAlso rdf:resource="http://danbri.org/foaf.rdf"/>
</foaf:Person>
</foaf:knows>

<foaf:based_near rdf:resource="http://dbpedia.org/resource/Milton_Keynes"/>

<PersonalProfileDocument rdf:about="">
<cc:license rdf:resource="http://creativecommons.org/licenses/by-nc/3.0/"/>
<dc:title>Tim Berners-Lee's FOAF file</dc:title>
<maker rdf:resource="http://www.w3.org/People/Berners-Lee/card#i"/>
<primaryTopic rdf:resource="http://www.w3.org/People/Berners-Lee/card#i"/>
</PersonalProfileDocument>

Linked data is link + description
• Description terms defined by lightweight ontologies.
• Designing lightweight ontologies is an art, not a science.
• Often, the formal ontologies are quite minimal, eg FOAF refuses to say
anything more than the class People is all people, and Agent is "things that
make stuff happen". Still, it does specify that People is an rdfs:subClass of
Agent. In practice, meaning is specified by use (in the actual links).
• It is considered good form to re-use existing relations, classes and
properties whenever possible. But this can set up some tension with the
meaning-as-use observation.


Linked data is here, now

Collectively, the data sets consist of over 13.1 billion RDF triples (dataset statistics),
3/23/2010
© which2010 Margaret Warren,around 142 million RDF links (linksset statistics) (November 2009).
2009 - are interlinked by Pat Hayes, CARMA Productions

Linked Image data?
• There are many images in existing linked data, but almost always as illustrations of things
(people, places, buildings..), as in FOAF.
• We want to describe images as objects in their own right. Images are our topic. So we need
an image-describing vocabulary with its own lightweight ontology.

Earlier work on image description
all focuses either on the things
depicted, or on the cultural setting
of the images, or on the curation
and cataloging of museum
collections.


Very lightweight image ontology

Questions one might ask….
1. Can any Thing be depicted? Do we need a class of 'depictable things'?
2. Do all images depict something?
3. What exactly is depiction anyway?
4. What exactly is an Image? (Is it a file type, for example? Are all images
digital images?)
etc…


• Dense lines of particular colors, splatters, all pictorial elements
• No distinct background, nothing depicted

Ontology can be molded to fit

A depiction of the
torso of
a female mannequin

AND/OR

A Pictorial Element
that looks like a
female torso


Depicting vs. looking like

• Simple depiction (as in FOAF) means something like "is a
picture of". Our depiction is wider in scope, and does not imply
visual similarity. An Xray image is a depiction, for example.
So we also need the notion of looking like.
• This is also important for characterizing pictorial elements.

looksLike
Depicted female Depicted female looksLike
iconic female
torso, x-ray torso female torso*
© 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions torso
3/23/2010

Images are pictorial elements.
If an image can depict, then a PART of an image can depict.
if a whole image can depict, so can part of an image.


Entire image is a pictorial element Entire image is a pictorial element
that looks like a female torso that depicts a female torso


Depiction is in the eye of the …

depicts Marie-Therese Walter depicts female torso

Defining classes of image
• Can now define for example 'line drawing': an image all of
whose component visible pictorial elements is a line.
<owl:Class rdf:ID="LineDrawing">
<owl:intersectionOf rdf:parseType="Collection">
<owl:Class rdf:about="#Image" />
<owl:Restriction>
<owl:onProperty rdf:resource="#hasVisualPart" />
<owl:allValuesFrom>
<owl:unionOf rdf:parseType="Collection">
<owl:Class rdf:about="#ImplicitPictorialElement" />
<owl:Class rdf:about="#LinearPictorialElement" />
</owl:unionOf>
</owl:allValuesFrom>
</owl:Restriction>
</owl:intersectionOf>
</owl:Class>


A Lightweight Ontology for Linking Images

Recomendados

Recomendados

Más contenido relacionado

Último

Último (20)

Destacado

Destacado (20)

A Lightweight Ontology for Linking Images