The MediaMixer project and community promote the use of semantic technologies for media mixing through real use cases and demos that showcase them, including Digital Asset Management systems. A typical MediaMixer demo will involve fragmenting media assets, annotating them using semantic descriptions and exposing these descriptions to customers, for fragment level search and selection. Fragments will be also linked to rights information based on a copyright ontology, which integrates contracts, policies and rights expressions based on existing standards like DDEX, Creative Commons or MPEG-21.

1. Roberto García
Universitat de Lleida, Spain
Createasphere Digital Asset Management Conference (DAMNY’13)
October 7-8, 2013, New York

2. 07.10.2013 Slide 2 of 25
MediaMixer
Community set-up and networking for the
reMIXing of online MEDIA fragments
• European Research Project
– Started November 2012
– Facilitate a Web-wide market for media
fragments re-purposing and re-selling
A history of G8 violence (© Reuters) http://slideshare.net/troncy/a-semantic-multimedia-web-create-annotate-present-and-share-your-media

3. 07.10.2013 Slide 3 of 25
Introduction
• MediaMixer promotes the use of
semantic technologies for media mixing
• Apply them in real use cases and demos
– Fragmenting media assets
– Annotating them using semantic descriptions
– Exposing descriptions for fragment level
search
–Representing, integrating and
checking copyright

4. 07.10.2013 Slide 4 of 25
Use Case
• User Generated Content

5. 07.10.2013 Slide 5 of 25
Use Case
• UGC from reused media

6. 07.10.2013 Slide 6 of 25
Use Case – ContentID
ContentID ©
©ContentID
match
match
match

7. 07.10.2013 Slide 7 of 25
Use Case
©
Block
Choices:
Track
Monetize
“Gangnam Style”, plus +33,000 videos using it, more than
$870,000 ad revenue - New York Times, Dec 7th 2012

8. 07.10.2013 Slide 8 of 25
Use Case - Disputes
• What if more than one ownership claim?
• If disputed, no one can monetize
©© match match

9. 07.10.2013 Slide 9 of 25
Use Case - Claim?
• Are we sure we can claim?
• Do we own that particular copy?
• In that territory?
• Also streaming on YouTube?
• Does the talent authorise YouTube?
• …

10. 07.10.2013 Slide 10 of 25
Use Case - Claim?
…1000s of pieces of registered content…
…1000s of videos on YouTube…
• Decision Support System (DSS):
– Disputes with Media on A and B, can we claim?
– Trade A for B with OtherMedia
(both win, start to get revenue)
A B

11. 07.10.2013 Slide 11 of 25
Use Case - Claim?
• Rights DSS requirements:
– Fine grained
– Scalable (largely automatized)
– Takes into account:
• Contracts
“…all rights on the live version but studio version just in the
US.”
• Policies
“…artist does not want his music together with violent
images”.
• Rights Expression Languages
DDEX metadata:<UseType>OnDemandStream</UseType>
<TerritoryCode>Worldwide</TerritoryCode>

12. 07.10.2013 Slide 12 of 25
MediaMixer
Technologies
Proposed Architecture
RDF Store
Rhizomer
Media
Explorer
UI generation
and user
assistance
UI generation
and navigation
Store license
or policy
Semantic
Media
Annotation
Rights Builder
User Interface
SPARQL
Rights Language
Mapping
DDEX
to RDF
DDEX
to RDF
DDEX
to RDF
Load licenses Load media
fragments annotations

13. 07.10.2013 Slide 13 of 25
Media Fragments
13
• Goal: transform whole videos to sets of
meaningful, indexable and re-usable video
fragments
Person, Snow, Trees,
Building, Ski, …
Baseball, Throwing, Sports,
Plant, Running, …
Kitchen, Indoor, Cake, …
…
Concept
and Event
Detection
Semantic
Media
Annotation

14. 07.10.2013 Slide 14 of 25
Linked Data
• Globally unambiguous
identifiers for any
concept needed
– Ideally more information
about the concept itself
is available via the
identifier
– Use of Web URIs →
Linked Data concept
space
– Would allow for
inference of concept
type and relationship to
other concepts
Semantic
Media
Annotation

15. dbpedia:Category:Whisky
dbpedia:Category:Distilled_beverages
broader
dbpedia:Category:Alcoholic_beverages
broader
Policy: “Avoid streaming media
showing alcoholic beverages
in countries prohibiting alcohol”
dbpedia:List_of_countries_with_alcohol_prohibition
dbpedia:Category:Member_states_of_the_Organisation_of_Islamic_Cooperation
dbpedia:Saudi_Arabia
subject
ddex:OnDe
mandStream
iso3166-2:SA
same as
location
theme
Check: streaming in Saudi Arabia?
Media Fragment
Semantic
Media
Annotation

16. 07.10.2013 Slide 16 of 25
From Controlled Vocabularies
…to Ontologies
INCREASING EXPRESSIVENESS
Features
Controlled
Vocabularies
Synonyms Taxonomies Thesaurus Ontologies
Control
Ambiguity
X X X X
Control
Synonym
X X X X
Hierarchical
Relations
X X X
Associative
Relations
X X
Custom
Relations
X

17. 07.10.2013 Slide 17 of 25
Copyright Ontology
Data & Ontologies Integration
RDF Store &
Reasoner
Copyright Ontologyschema.org
DDEX Ontology
ISO 3166
Country Codes
Place
Action
SA
locationrange
is a
CreationObject
VideoScreenCapture
TVEpisode subclass of
subclass of
Use
MakeAvailable
subclass of
OnDemandStream
subclass of

18. 07.10.2013 Slide 18 of 25
Copyright Ontology
Lifecycle, Actions, Rights,…
Creator
Actor
Producer
Broadcaster
User
Motion Picture
Script
Adaptation
Performance
manifest
perform
record
Communication
broadcast
transform
Literary Work
tune
Copyright
EconomicRights
RelatedRights
MoralRights
DistributionRight
ReproductionRight
PublicPerformanceRight
FixationRight
CommunicationRight
AttributionRight
TransformationRight
IntegrityRight
DisclosureRight
WithdrawalRight
PermorfersRights
ProducersRights
BroadcastersRights
RentalRight
ImportationRight
SoundRecordRight
MotionPictureRight
BroadcastingRight
MakingAvailableRight
AdaptationRight
TranslationRight
RDF Store &
Reasoner

19. 07.10.2013 Slide 19 of 25
• Ontology provides building blocks to model and reason
about contracts, policies, rights expression languages…
No, because it is matched
to pattern
which prohibits it:
“Artists does not want her
music with violent images”
Can we claim asset X
Copyright Ontology
Reasoning
RDF Store &
Reasoner

20. dbpedia:Category:Whisky
dbpedia:Category:Distilled_beverages
broader
dbpedia:Category:Alcoholic_beverages
broader
Policy: avoid streaming media
showing alcoholic beverages
in countries prohibiting alcohol
dbpedia:List_of_countries_with_alcohol_prohibition
dbpedia:Category:Member_states_of_the_Organisation_of_Islamic_Cooperation
dbpedia:Saudi_Arabia
subject
iso3166-2:SA
same as
REASONING
ddex:OnDe
mandStream
location
theme

21. 07.10.2013 Slide 21 of 25
Semantic Data Exploration
http://rhizomik.net/mediamixer/
Rhizomer
Media Explorer

22. 07.10.2013 Slide 22 of 25
http://rhizomik.net/mediamixer/rightsbuilderui/
Assisted Policy and Agreement
Modelling User Interface
Rights Builder
User Interface

23. 07.10.2013 Slide 23 of 25
Semantic Technologies for
Rights Data Integration & Intelligence
MediaMixer
Semantic Techs
Policy 1:
The licensor
disagrees the
licensee
adaptsmedia
fragments
containing…
<xml>
<right>
<play/>
</right>
<content
id=“frg1”/>
</xml>
Semantic
Media
Annotation
Rights Builder
User Interface
Rights Language
Mapping
RDF Store &
Reasoner
media fragments,
semantic data,
annotation, ontologies,
reasoning,…

24. 07.10.2013 Slide 24 of 25
Interested? Get involved
• Join MediaMixer community to know more
(http://community.mediamixer.eu):
– discussions, use cases, demonstrators, tutorials,
presentations, software,…

25. More details:
http://community.mediamixer.eu
@project_mmixer
Roberto García (@rogargon)
Universitat de Lleida, Spain
http://rhizomik.net/~roberto
Thank you for your attention