RUCoD is a formal representation of a content object, consisting of descriptions of media types associated.The acronym stands for Rich Unified Content Description. Released in its final version in the iSearch project (see http://www.isearch-project.eu/isearch/RUCoD/) it will be used and extended in CUbRIK.
08448380779 Call Girls In Friends Colony Women Seeking Men
RUCoD in CUbRIK at Star2013
1. 1
Rich Unified Content
Description (RUCoD)
Vincenzo Croce
Research & Development
Laboratory
STAR 2013 Lausanne, 18-19 January 2013
2. Objectives 2
To develop a formal description for each type of multimedia
content (text, audio, image, video and 3D content)
To develop formal descriptions for real world and social
information
To clearly specify the format of the Rich Unified Content
Description (RUCoD)
To represent in the same format, the actual content
(multimedia information) along with the additional contextual
information (real world, user-related)
STAR 2013 Lausanne, 18-19 January 2013
3. The Concept of Content 3
Object
“A Content Object is the representation of a specific
instance of either a physical object or a physical entity (an
entity that has physical existence), which might have
multiple views (many images, videos, audio files, text, real-
world and user-related information).”
Similar approaches:
Multimedia Document (MMD): a set of co-occurring multimedia
objects (e.g. images, audio and text) that are of different modalities
but carry the same semantics. If two multimedia objects are in the
same MMD, they can be regarded as context of each other.
Multimedia Bag: defines a container including text instances, image
instances and audio instances that share the shame semantic
concepts
STAR 2013 Lausanne, 18-19 January 2013
4. The Concept of Content 4
Object
A CO may consist of several
media types, user-related
information and real-world
information.
A CO can be the result of an
authoring process (e.g. using
an authoring tool).
RUCoD is a formal
representation of a CO
consisting of descriptions
of various media types
somehow associated to
each other.
STAR 2013 Lausanne, 18-19 January 2013
5. Example Content Object 5
Great (Cheops) –
3D object
Pyramid of Giza Image
CO:
Great (Cheops) –
Pyramid of Giza
Text: Real-world (location)
It is believed the pyramid was built
as a tomb for Fourth dynasty
Egyptian pharaoh Khufu(or
Cheops) and constructed over a
14 to 20 year period concluding
STAR 2013 Lausanne, 18-19 January 2013 2560 BC…
around
6. RUCoD Specification 6
Real World Descriptors
Position Weather
Date Time
(GPS, etc) (temperature)
Sensors …
Header
CO ID Creator Version CO Types Multimedia RW-info U-Info …
Header Low-Level Descriptors Real World User-related Descriptors
Descriptors
Low-Level Descriptors
Text-based Image Video Audio 3D
descriptors descriptors descriptors descriptors descriptors …
User-related Descriptors
Expressions Emotions Valence Arousal …
STAR 2013 Lausanne, 18-19 January 2013
12. RUCoD Schema 12
• RUCoD Schema Final Version (1.4.1) released in
November 2011:
• The RUCoD.xsd Schema file
• The RUCoD_Descriptors.xsd Schema file
http://www.isearch-project.eu/isearch/RUCoD/
STAR 2013 Lausanne, 18-19 January 2013
13. Block diagram of the I-SEARCH framework 13
STAR 2013 Lausanne, 18-19 January 2013
14. Authoring & Content Analytics
14
Architecture
Authoring
Content Analytics
STAR 2013 Lausanne, 18-19 January 2013 Indexing triggering
15. RUCoD in I-SEARCH Use 15
Cases Social retrieval
UC2: UC3: Furniture retrieval
of music •Text
•Audio (music) •3D objects
•Text •Images
•Video clip •Real-world (location, time)
UC1: Music retrieval •Real-world (location)
•Audio (music) •Emotions
•Text UC4: Search for Multimedia
using smartphone
•Images
Rich •Text
•Real-world
Unified •3D objects
(location, time) Content •Images
•Emotions
Description •Real-world (location, time)
UC7: Game avatar retrieval
•3D objects UC5: Search for specific
•Images product
•Text
•Video UC6: 3D game component •3D objects
•Emotions retrieval
•Images
•Text
•Video
•3D objects
•Audio (sounds)
•Images
•Real-world (location, time)
•Audio (sounds)
•Emotions
•Real-world
STAR 2013 Lausanne, 18-19 January 2013 (location, time)
16. Comparison with MPEG-7 16
What we use:
• MediaLocator and MediaUri are used to describe the link
to a specific media item.
• Creator is used for description of the author of a media
item.
• Annotation as a part of RUCoD represents textual
information of a media item or CO.
• Image/Video/Audio Descriptors are used for the low-level
descriptions of the separate media items within a CO.
• Segment is used to describe a temporal video segment.
STAR 2013 Lausanne, 18-19 January 2013
17. Comparison with MPEG-7 17
What we adapted:
• ContentObjectName, ContentObjectCreationInformation
instead of name and CreationInformation to represent the
name and creators of COs.
• TextDescription, Shape3DDescription, ImageDescription and
VideoDescription, similar to MPEG-7 ContentDescription to
distinguish between the descriptors of different modalities
inside the same RUCoD.
STAR 2013 Lausanne, 18-19 January 2013
18. Comparison with MPEG-7 18
What is new:
• Cross-modal & multimodal retrieval are not entirely supported by the
standards.
• New types of information describing the COs are introduced, such as real-
world descriptors and user-related descriptors. These enrich the CO
description and improve the retrieval performance, by introducing new
querying capabilities.
• With respect to low-level descriptor extraction for media items, novel
descriptors are introduced. As an example, for 3D content description, new
state-of-the-art descriptors are introduced, which achieve higher retrieval
performance than those included in MPEG-7. Similarly, new descriptors are
introduced for image, video and audio content.
• The low-level description of media items is also accompanied by
specification of the matching scheme for each descriptor. In this case, the
description scheme does not leave the responsibility for choosing the
appropriate matching method to the search engine.
STAR 2013 Lausanne, 18-19 January 2013
19. Comparison with JPSearch
What is similar/different:
• JPSearch is designed in a way that decouples the
components of image search and provides a standard
interface between these components. Its aim is to build a
standard for interoperability among image search and
retrieval systems
• RUCoD specification is focused on the description of
COs and it addresses a broad range of media (apart
from images), real-world and user-related information
STAR 2013 Lausanne, 18-19 January 2013
20. Comparison with MPEG-21
What we use:
• A multimodal approach to media (which can be of any type)
• Allowing the creation and attachment of rich metadata to
digital objects
STAR 2013 Lausanne, 18-19 January 2013
21. Comparison with MPEG-21
What is similar:
• Ability to create multimedia content objects (Digital Items in
MPEG-21)
• Ability for content adaptation, although achieved differently
(e.g. through FileFormat elements in the RUCoD)
• L-Descriptors and R-Descriptors of RUCoD could be attached
to MPEG-21 objects although not directly foreseen by the
standard
STAR 2013 Lausanne, 18-19 January 2013
22. Comparison with MPEG-21
What is different/new:
• The CO broadens the concept of Digital Item making it more
general and flexible
• Unification of the actual metadata and descriptors (e.g. L-
Descriptors) together with real world and user-related parts in
the same format.
• RUCoD is particularly targeted at indexing, sharing, search
and retrieval
• RUCoD overcomes the traditional hierarchical object model
(also foreseen in MPEG-21) allowing for more flexible and
user-centric 'connections' (e.g. RelatedSemanticConcepts
field)
STAR 2013 Lausanne, 18-19 January 2013
23. Ongoing Work
• RUCoD initially designed to serve the needs of the I-
SEARCH framework
• However, it is not mature enough to be used in a wider
range of applications.
• RUCoD will be extended within the EU-funded project
CUbRIK (CERTH and ENG are participants).
• RUCoD will be adopted in CUbRIK.
• A first attempt: SMILA Hackathlon, November 2011,
Keiserslautern.
• RUCoD was presented in the workshop
• RUCoD was used as descriptor scheme to check indexing and search
within the SMILA engine
STAR 2013 Lausanne, 18-19 January 2013
24. 24
Questions?
STAR 2013 Lausanne, 18-19 January 2013