03 synchronized distance learning

Synchronized Distance
Learning
Timothy K. Shih

Outline
• White Board and Chat Room
– Designing Shared Whiteboard
• Multimedia Synchronization
– Microsoft ASF
– SMIL
• Synchronization Models
– Petri Net
– Interval Temporal Logic (optional)

White Board and Chat Room
• Chat Room
– Almost Everywhere
• White Board
– On-Line Annotation with Floor Control

Text
Acknowledgement
Timothy K. Shih and Yi-Chun Liao, MINE Lab, Tamkang University, Taiwan
• Text and simple graphic drawing
• Annotation message broadcasting via multiple uni-casting
• NetMeeting 3.0, COM (Component Object Model), ActiveX
control
• Acknowledgement from Remote Stations is included
• Floor Control: Control Modes, Channel, and Group
On-Line Annotation

Request Button
Request to SpeakErrorAcknowledgement
Question Button
Speaking
Return Control
Interface and Demo

Demo
Annotation with Feedbacks

• 30 Client connections
• Heterogeneous Network
Demo
Video

• The control of ―Who to Speak‖ in a communication
environment
• Concepts:
– Channel: communication modal (text, audio, video, etc.)
– Mode: communication style of a channel (Free Access vs. Equal
Right)
– Group: the IP domain of a channel
• Example: 4 Channels, 3 Groups, 10 Persons, 2 Modes
Video Conferencing (ER) and
Chat Room Tool (FA)
Chat Room Tool (FA)
Annotation Tool (ER)
Floor Control

Chat RoomAnnotation
Virtual PanoramaShared Notebook
Video
Audio
Video-on-Demand
Multi-modal and Multi-
channel Communication

• Enhance the clearness of
video presentation
• Add bitmapped animation
to the video
Augmented Video
Conferencing Tool

Augmented Video Presentation
Screen Capture
Video Control
Video + Audio
MPEG-2

Implementation
• Configuration
• Five Control
Modes
– Normal
– Resized
Screen
– Resized Video
– Full Video
– Full Screen

• Audio and Video
Devices and Codec
• Screen and CCD
Camera Resolutions
Device and Layout Configuration

• Problems to Solve
– Distance Precision –
Space Discontinuity
– Video Precision –
Eye Contact
– Object Tracking
• The instructor
• The whiteboard
– Interaction Protocol
• Floor Control
– Performance
Challenges

Multimedia Synchronization
• Why Multimedia Synchronization
• Synchronization Methods
– Microsoft ASF
– SMIL
• Synchronization Models (Theories)
– Petri Net
– Interval Temporal Logic

Video
Music
Audio
Slide
Message
Annotation
ASF Records
Timothy K. Shih, Y. R. Liu and Sheng-En Yeh, MINE Lab, Tamkang University, Taiwan
• Automatic Recording
• Use Commercial Codec and the Advanced Streaming
Format (ASF) from Microsoft
• Video and Slide Synchronization
• Text and Drawing Annotation Synchronization
• Slide Selection
• Presentation Editing
• Automatic Loading
Multimedia Synchronization
Using ASF

• Event markers are embedded in a ASF file
• Arbitrary activation of markers
• Do not record the entire stream as one video clip
• Video streaming depends on Codec
Video Stream
Text
User Interrupt User Interrupt User Interrupt
Slide Slide Slide
 MS Office XP – Presentation Broadcast provides video
and slide synchronization (PowerPoint and IE)
Event-based Synchronization

Choose Device
Choose Codec
Presentation Recording Tool

Select Output
Slide Selection
Presentation Playback Control Functions
Presentation Tool

• Hard to record the entire lecture without
making a mistake
• Different presentation needs different level of
details
• Lecture summarization and editing
• Atomic presentation object – a PowerPoint
Slide (with its video and annotation events)
Text
Atomic
Object
Adaptive Presentation

Three Levels
of Details
Automatic Loading Presentation Editing
Multi-Level Presentation

Synchronized Multimedia
Integration Language
(SMIL)
• Recommended by W3C
• XML-based language
• Designed for Internet use
• Easy to write similar to HTML
• Integrates several media formats
• Plain text documents with a ―.smi‖ or
―.smil‖ suffix
Part of slides about SMIL are summarized from:
http://service.real.com/help/library/guides/production/realpgd.htm

History
• Dec 1995 - SMIL designing started
• Nov 1997 - SMIL 1.0 Recommendation
• Mar 1998 - First SMIL implementation
• Aug 1999 - First public draft of SMIL 2.0
• Aug 2001 - SMIL 2.0 Recommendation

Support for SMIL in
Players / Browsers
2.0 • X-Smiles 0.40: SMIL 2.0 Basic Profile
2.0 • Oratrix GRiNS player: SMIL 2.0 Language
2.0 • IE 5: many SMIL 2.0 modules
1.0 • QuickTime Player 4.1
1.0 • RealPlayer 8
1.0 • Compaq HPAS
1.0 • Productivity Works Lp Player
1.0 • SOJA from Helio
1.0 • S2M2 by NIST
1.0 • Schmunzel by SunTREC Salzburg

Use Cases for SMIL
• Slideshows
• Advertisements
• Internet TV
• Education
• Corporate communications
• Product information
• User’s Guides
• Net meetings
• etc.

General Rules
• Documents start with <smil> and end with
</smil>
• <head> section, which is optional; and a
<body> section which is required
• Tags and Attributes in lowercase only
• All elements must have closing tags or /
• Attribute values must be enclosed in double-
quotes
• HTML-style comments are allowed.
–

Typical SMIL Elements
<head> - Head element
<meta> - Meta data, such as author, copyright...
<layout> - Layout of the presentation
<region> - Region describing positioning
<body> - Body element
<seq> - Sequential time container
<par> - Parallel time container
<img>, <audio>, <video> - Media to be played
<switch> - Selects between elements
<a>, <area> - Linking

A SMIL Example
<smil>
<head>
<layout>
<region id=”reg1” top=”0” left=”0” width=”300” height=”200” />
<region id=”reg2” left=”300” width=”300” height=”200” />
</layout>
</head>
<body>
<seq>
<img id=”id1” src=”intro.jpg” region=”reg1” dur=”4s” />
<audio id=”id2” src=”music.wav” dur=”2s” />
<par>
<video id=”id3” src=”movie5s.mpg” region=”reg1” />
<img id=”id4” src=”intro.jpg” begin=”3s” region=”reg2” />
</par>
</seq>
</body>
</smil>

<head> element
• Information not related to the temporal
presentation of the material
• May contain the following children
– <meta>
– <switch>
– <layout>

<layout> element
• Probably the most important element
• Specify the layout of the document, as
well as the layout of the multimedia
elements in the document
• If not specified, how document looks will
depend on the player
• Two children allowed here
– <root-layout>
– <region>

<root-layout> element
• An empty element
• Defines the size of the main window
• You cannot specify media elements in
here
• There can only be one <root-layout>

<root-layout> attributes
• background-color
– Can use named or hexadecimal values.
• height, width
– All specified in pixels
• title

<region> element
• Areas within <root-layout> in which
media elements will appear
• May define as many as you want
• Very similar to frames in HTML
• Are allowed to overlap — controlled by z-
order
• Any regions that lay outside the <root-
layout> will be cut off

Required attributes
• id
– A unique text name for the region
• top, left, width, height
– Pixels or percentages or mixed

Example
• <head>
<root-layout width="480" height="300"/>
<region id=“someid" top="60" left="120"
width="240" height="180"/>
</head>
Screen
(60, 120)
 240 
180

<body> element
• Information related to the temporal and linking
behavior of the document
• Children include
– Synchronization elements
• <par>, <seq>
– Media elements
• <animation>, <audio>, <img>, <text>, <textstream>,
<video>, <ref>

Synchronization elements
• <par> element — children of a <par>
element occur in parallel
• <seq>element — children of a <seq>
element appear in sequential order
• Valid children of <seq> and <par>
elements include
– <seq> & <par>
– Any of the media elements

Combining <seq> and <par> tags
<seq>
clip 1
<par>
clip 2
clip 3
</par>
clip 4
</seq>
<par>
clip 1
<seq>
clip 2
clip 3
</seq>
clip 4
</par>
1
2
3
4
3
1
2
4

Media elements
• The various pieces of multimedia that go into the
SMIL document
• Continuous media
– <animation>, <audio>, <textstream>, <video>
– Have an intrinsic duration
• Discrete media
– <text>, <img>
– Have no intrinsic duration
• The actual tag name doesn’t matter
• Type attribute is used to determine media type
• Could use <ref> tag

Media element attributes
• All media elements have certain
attributes in common
• You use these attributes to specify how
the children of <seq> and <par>
elements will appear in the presentation

Specifying media and
location
• src
– Specifies the source of the media element
– Identical to the src attribute of the <img> tag
in html
• region
– The area of the root-layout in which the
media element is to be displayed
– e.g., <video src="hal.mov" region="video_region" />
– If you don’t specify a region, it’s
implementation dependent

Setting begin and end time
• begin & end attributes
• Specifies the explicit beginning and
ending time of the element
– e.g., <video src="hal.mov" begin="10.5s" end="45.5s"
/>
• Relative to parent element regardless of
own length
• May use with <par> and <seq> elements
to affect entire group

Setting internal begin & end
times
• clip-begin & clip-end attributes
• Specifies the internal beginning and
ending time of a clip
– E.g., <video src="hal.mov" clip-begin="2.5s" clip-
end="15.8s" />
• May combine with begin and end
attributes

Setting a duration
• dur attribute
• Explicit duration of the media element
• Time value, or ―indefinite‖
• Should not use dur with end attributes

Filling up time
• fill attribute
– Specifies what happens to the clip when it is
done
– Freeze or remove
• repeat
– Specifies how many times the clip should
repeat itself
– Any integer is valid
– Can add repeat attribute to the <par> and
<seq> elements

Synchronizing elements
• By default, <par> elements end when all
elements finish
• Can be modified with the end attribute to
force an end time for the <par> element
• Or you can use the endsync attribute of
the <par> element to synchronize the
ending of its clips
• Three values for the endsync attribute
– first, last, id(element id)

Linking
• SMIL provides two ways to link to other
documents
• <a> tag
– Identical to the <a> tag in html
– Specify an href attribute with a URL
– Can surround and turn an entire media
element into a link
• <anchor> tag
– Defines a ―hot spot‖ inside a media element
– Similar to HTML image maps

<anchor> tag
• Specified as a child of a media element
– <video src="video.rm" region="video_region">
<anchor href="url here" coords="20, 40, 80, 120">
</video>
• Where the coords attribute specifies the
location within the media element that is
the hot spot
• May use pixels or percentages

Control Structure
• SMIL provides a <switch> tag
• Can use it to allow player to choose one
of several test-attributes
• Format
<switch>
<choice1 test-attribute="value1"/>
<choice2 test-attribute="value2"/>
…
</switch>
• Various test-attributes have been defined

Using <switch> for
language
• You can use the <switch> tag to allow player
to choose the appropriate language
• <par>
<video src="video/seattle.rm" />
<switch>
<audio src="french/seattle.rm" system-language="fr"/>
<audio src="german/seattle.rm" system-language="de"/>
<audio src="spanish/seattle.rm" system-language="es"/>
<audio src="english/seattle.rm"/>
</switch>
</par>

Using <switch> for system
bitrates
• <switch>
<par system-bitrate="75000">

<audio src="audio/newsong1.rm"/>
<video src="video/newsong1.rm"/>
</par>

</par>

</par>
</switch>

For information
• W3C’s Synchronized Multimedia Group
– http://www.w3.org/AudioVideo
• Just SMIL
– http://smw.internet.com/smil

Petri Net
• What are Petri Nets
– A tool for system study
• System Modeling
– Structure
– Dynamic Behaviors
• System Evaluation
Source: Feng-Cheng Yang
These slides are modified based on Feng-Cheng Yang’s presentation

Using Petri Net
• Modeling
– Representation for a System
– Often in Mathematical Terms
– Manipulated to reveal information
• Example of Discrete System Analysis
– Bank System
• Teller Components
– Idle state
– Busy state
• Customer Components
– Idle state (waiting in the line)
– Busy state (being served)

Multimedia Presentation
SMIL
<par>
clip 1
<seq>
clip 2
clip 3
</seq>
clip 4
</par>
3
1
2
4
Petri Net (simplified)
3
1
2
4

Features of a System
• Components
– Subsystems
– Separated and Interacting
• States
– Depend on the past history
– Change over time

System Behaviors
• Concurrency (Parallelism)
– Activities of different components occur
simultaneously
– Difficult to model
• Sequencing
– Serial activities occur one by one
between components
– Easy to model

History
• Basic concept was created in 1962 by Carl
Adam Petri in his doctoral thesis
(W. Germany)
– Communication between asynchronous
components of a computer system
• Early theory, notation, and representation
are developed by the Information System
Theory Project (Applied Data Research, US)
– Conduct by A.W. Holt (1968)

History (cont.)
• Considerable research and publication in
1970
– Directed by Prof. Jack B. Dennis at M.I.T.
• Conference on Petri Nets and Related
Methods in 1975 at M.I.T.
• Workshop on Petri Nets held in Paris in
1977
• In 1979 an advanced course on General Net
Theory is held in Hamburg
• A Special Interest Group on Petri nets was
formed in Germany

Petri Net Structure
(Mathematical Model)
• Four parts
– P: a set of places
– T: a set of transitions
– I: an input function
• A mapping from a transition tj to a collection
of places I(tj )
• The places are input places of tj
– O: an output function
• A mapping from a transition tj to a collection
of places O(tj )
• The places are output places of tj
Pi
tj
Pi
tj

Definition 1
• A Petri net structure is a four-tuple:
C = (P,T,I,O)
• P = { p1, p2,…, pn }, n ≥ 0
• T= { t1, t2,…, tm }, m≥0
• P∩T = ∅
• I:T→P∞, a mapping from transitions to bags
of places
• O:T → P∞, a mapping from transitions to
bags of places

Petri Net Representation
• A place pi is an input place of a transition tj
if pi ∈ I(tj)
• A place pi is an output place of a transition tj
if pi ∈ O(tj)
• A bag is a generalization of sets which
allows multiple occurrences of an elements
in a bag.
– E.g.: O(tj) = { p1, p1, p1, p2, p3, p4, p4, p5 }
Pi
tj
Pi
tj

Example 1
P2
P4P1 P5
P3
t1 t2
t3
t4

Example 1
• C = ( P,T,I,O )
• P = { p1 , p2 , p3 , p4 , p5 }, n = 5
• T = { t1 , t2 , t3, t4 }, m = 4
• I(t1 ) = { p1 } O(t1 ) = { p2 , p3 , p5 }
• I(t2 ) = { p2 , p3 , p5 } O(t2 ) = { p5 }
• I(t3 ) = { p3 } O(t3 ) = { p4 }
• I(t4 ) = { p4 } O(t4 ) = { p2 , p3 }
P2
P4P1 P5
P3
t1 t2
t3
t4

Multiplicity
• #(a, A) : The number of occurrences of
element a in bag A.
• The multiplicity of an input place pi for a
transition tj is #( pi, I(tj) )
• A transition tj is an input of a place pi, if pi is
an output of tj
• A transition tj is an output of a place pi, if pi
is an input of tj
Pi
tj
Pi
tj
# = 3

Definition 2
• Input and output functions are extended as
follows:
– I:T→P∞, O:T → P∞ , such that
• #( tj, I(pi) ) = #( pi, O(tj) ) and
• #( pi, I(tj) ) = #( tj, O(pi) )
Pi
tj
Pi
tj

Example 2
P2
P4P1
P5
P3
t1
t2
t3
t4
P6
t5

Example 2
• C = ( P,T,I,O )
• P = { p1 , p2 , p3 , p4 , p5 , p6 }, n = 6
• T = { t1 , t2 , t3, t4, t5 }, m = 5
• I(t1 ) = { p1 } O(t1 ) = { p2 , p3 }
• I(t2 ) = { p3 } O(t2 ) = { p3 , p5 , p5 }
• I(t3 ) = { p2, p3 } O(t3 ) = { p2, p4 }
• I(t4 ) = { p4 , p5 , p5 , p5 } O(t4 ) = { p4 }
• I(t5 ) = { p2 } O(t5 ) = { p6 }
P2
P4P1
P5
P3
t1
t2
t3
t4
P6
t5

Example 3
P2
P4
P1
P5
P3
t1
t2
t3
t4
P6
t5
P7
P8
P9
t6

Example 3
• C = ( P,T,I,O )
• P = { p1 , p2 , p3 , p4 , p5 , p6 , p7 , p8 , p9 }, n = 9
• T = { t1 , t2 , t3, t4, t5 , t6 }, m = 6
• I(t1 ) = { p1 } O(t1 ) = { p2 , p3 }
• I(t2 ) = { p8 } O(t2 ) = { p1 , p7 }
• I(t3 ) = { p2 , p5 } O(t3 ) = { p6 }
• I(t4 ) = { p3 } O(t4 ) = { p4 }
• I(t5 ) = { p6 , p7 } O(t5 ) = { p9 }
• I(t6 ) = { p4 , p9 } O(t6 ) = { p5 , p8 }
P2
P4
P1
P5
P3
t1
t2
t3
t4
P6
t5
P7
P8
P9
t6

Petri Net Graphs
• Bipartite directed multigraph
• Vertices:
– Circles (places) and Bars (transitions)
• Links:
– Directed arcs (represent the relations
between transitions and places)
• Represented by functions I and O
• Multiple arcs directed from a place to
a transition is allowed

Definition 3
• A Petri net graph G is a bipartite directed
multigraph, G = ( V, A ), where
V = { v1, v2, …, vs } is a set of vertices and
A = {a1, a2, …, ar} is a bag of directed arcs,
ai = (vj , vk ), with vj, vk ∈ V
• The set V can be partitioned into two
disjoint sets P and T such that V = P ∪ T,
P ∩ T = ∅ , and for each directed arc,
ai ∈ A, if ai = (vj , vk ), then either
vj ∈ P and vk ∈ T or vj ∈ T and vk ∈ P

Definition 4
• Define V = P ∪ T.
• Define A as a bag of directed arcs such that
for all pi ∈ P and tj ∈ T ,
– #( (pi , tj ), A ) = #( pi , I(tj) ) and
– #( (tj , pi ), A ) = #( pi , O(tj) )
• G = ( V, A ) is a Petri net graph which is
equivalent to the Petri net structure
C = ( P, T, I, O )

The Dual of a Petri Net
• The dual of a Petri net C = ( P, T, I, O )
is the Petri net C’ = (T, P, I, O ) which
results from interchanging places and
transitions

The Dual of Example 1
P2 P4P1
P3
t1 t5
t3
t4
t2
P2
P4P1 P5
P3
t1 t2
t3
t4

The Inverse Petri Net
• The inverse Petri net for a Petri net C
= ( P, T, I, O ) is defined by
interchanging the input and output
functions,
-C = ( P, T, O , I ).

The Inverse of the Dual of Example 1
P2 P4P1
P3
t1 t5
t3
t4
t2
P2 P4P1
P3
t1 t5
t3
t4
t2

Petri Net Markings
• A marking μ is an assignment of
tokens to the places of a Petri net
• The number and position of tokens
may change during the execution of a
Petri net
• The tokens are used to define the
execution of a Petri net

Definition 5
• A marking μ of a Petri net C = ( P,T,I,O ) is a
function from the set of places P to the
nonnegative integers N; μ : P → N
• The marking can also be defined as an n-vector,
μ = [ μ1 μ2 …μn ]
• The number of tokens in place pi is μi, i =
1, …, n
– μ(pi ) = μi

Marked Petri Nets
• M = ( C, μ ) is a Petri net structure and a
marking μ
– M = (P,T,I,O, μ )
• Tokens are represented by small dots in the
circles which represent the places of a Petri
net
• The number of tokens assigned to a place
is unbounded
• The number of possible markings of a Petri
net is infinitive

A Marked Petri Net of Example 1
• The marking μ = [ 1 2 0 0 1 ]
t1 t2
t3
t4
P1 P5
P2
P4
P3

Another Marked Petri Net of Example 1
• The marking μ = [ 0 2 0 5 1 ]
t1 t2
t3
t4
P5
P2
P4P1
P3

Execution Rules for Petri Nets
• Tokens reside in places and control the
execution of the transitions of the net
• A Petri net executes by firing transition
• A transition fires by removing tokens from
its input places and creating new tokens
which are distributed to its output places
– The tokens are not moving from one place to
the next place
– They are consumed by the transition and
regenerated from the transition to the next
place
• A transition may fire if it is enabled

Execution Rules for Petri Nets (Conti.)
• A transition is enabled if each of its input
places has at least as many tokens in it as
arcs from the place to the transition
• The tokens in the input places which enable
a transition are its enabling tokens
• A transition fires by removing all of its
enabling tokens from its input places and
then depositing into each of its output
places one token for each arc from the
transition to the place

Definition 6
• A transition tj ∈ T in a marked Petri net
C = ( P,T,I,O ) with marking μ is enabled if
for all places pi , μ(pi) ≥ #( pi, I(tj) )
tj
1 token ≥ one arc
3 tokens ≥ three arcs
2 tokens ≥ two arcs

Definition 7
• A transition tj in a marked Petri net with
marking μ may fire whenever it is enabled
• Firing an enabled transition tj results in a new
marking μ’ defined by
• μ’(pi ) = μ (pi ) - #(pi , I(tj) ) + #(pi , O(tj) )
tj
Pi
tj
Pi
Remove 2 tokens (for 2 arcs)
Remove 3 tokens (for 3 arcs)
Add 1 token (for 1 arc) Add 1 token
(for 1 arc)

Firing Examples
• Example a:
– I(t3 ) = { p2} O(t3 ) = {p7, p13 }
– t3 is enabled whenever there is at least one
token in place p2
• Transition t3 fires by removing one token
from p2 and depositing one token in place
p7 and one token in place p13
t3 t3
P2
P7
P13
P7
P13
P7P2

Firing Examples (Cont.)
• Example b:
– I(t2 ) = {p21, p23} O( t2 ) = {p23, p25, p25 }
– t2 is enabled whenever there is at least one token in place
p21 and at least one token in place p23
– Transition t2 fires by removing one token from p21 and one
token from p23 and then deposits one token in place p23 and
two tokens in place p25
t2 t2
P21
P23
P25
P23
P25
P21

Firing Examples (Cont.)
• t1 , t3 , and t4 are enabled
t1 t2
t3
t4
P4
P1
P3
P5P2

• t4 is fired
t1 t2
t3
t4
P4
P1
P3
P5P2

• t1 is fired
t1 t2
t3
t4
P4
P1
P3
P5P2

• t3 is fired
t1 t2
t3
t4
P4
P1
P3
P5P2

Summary
• Petri Net Definitions
• Petri Net Examples
• For multimedia presentations
– Transitions are used for synchronization
– Places can represent multimedia resources
• Related to Interval Temporal Logic
3
1
2
4

Based on Petri Net
• Defining a operation model of multimedia
presentation by features of Petri Net
• Constructing an algorithm based on
Timed Petri Net for multimedia
presentations

Background
• Petri net is a graph and a mathematical tool
• Multimedia presentations includes timing
control and user interaction
• Describe multimedia objects by Timed Petri
Net
• Create new definitions based on Petri Net for
interactions
• Implement a tool for multimedia presentation
designs

An operation example of Petri net
Place
A rc
Tokens
Transition
A rc
A rc
Place
Place
Tokens
Place
Place
Place
A rc
A rc
A rc
Transition
Token
Token
Token
(a) A initial marking before transition firing (b) A new marking after transition firing

Interpretations of places and
transitions
Tansitions A ttributesInput Places O utput Places
Input Places Transition A ttributes O utput Places
D ata Processing Inform ation
R equest A vailable Service
R esource Perm ission A pproach
Precondition Event Postcondition
M ultim dia Tim ing Playing

Two special cases in Petri net
(a) A sink transition of Petri Net
(End of Presentation)
(b) A source transition of Petri Net
(Begin of Presentation)

Self-loop and Weights
1
1
1
2
1
(a) A self_loop of Petri Net (b) Weights of synchronous arcs of Petri Net

New Components for Multimedia Objects
: Place and token is for playing a m ultim edia resource.
: U ser transition is for accepting a m essage and causing
the activation of connected transitions.
: U se r arc is fo r connec ting fr om a user transition to a
transition.
: Sy nc hro nou s arc is for con ne cting fr om a p la ce to a
transition or from a transition to a place.
: Transition is for synchronous control.
New
New

Example of Multimedia Petri net
im age
text
video
sound
m usic
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
Syn. arc
U ser arc
U ser arc
navigation
m essage
navigation
m essage
user transition
transition
transition
transition
transition

Definitions of place, transition and user
transition in the Multimedia Petri net
O bject attributes
File form at
D uration
File nam e
Execution tool
File attributes
Place node
Transition
U ser Transition
A ttributes
M essages
Tim ing
O bject relations
Interrupt
Pause
Stop
C hange

Algorithm: Compute_Node_Time
• Construct a Transition_Graph reachable from one of the starting
• Navigation_Messages;
• Initialize the node pointed by the Navigation_Message to
• Node_Time = 0;
• Sort the Transition_Graph to a Transition_List by Topological sort;
• For each node N in the Transition_List, except the first,
• Begin
• For each incoming edge, compute the Estimated_Time =
• Node_Time + Edge_Duration;
• Set Node_Time of node N to the maximum of all Estimated_Time
• of incoming edges;
• End

Algorithm: Run_Petri_Net
– Begin
– For each node N in the Transition_List Do Begin
– Collect places pointed by Sync_Arcs from node N in Sync_Set(N);
– Sort the Transition_List according to Node_Time in a non-
– decreasing order;
– End
– CoBegin
– Process1:
– For each node N in the Transition_List Do Begin
– Sequentially play resources in Sync_Set(N) at Node_Time(N)
– concurrently;
– End
– Process2:
– LoopBegin
– If Process1 ends, Then Process2 ends;
– If there is a Navigation_Message of a User_Transition Then
– Exits with the next starting Navigation_Message;
– If there is a Selection, Assignment, or Condition Then
– Perform the action accordingly;
– LoopEnd
– CoEnd
– End
– Algorithm: Run_Presentation
– Begin
– LoopBegin
– If there is a starting Navigation_Message Then
– Compute_Node_Time;
– Run_Petri_Net;
– Else
– End Presentation
– LoopEnd
– End

A initial graph of Multimedia Petri Net
Text
V ideo
W ave
Sound
Bm p
M idi

Transition graph for simplified Multimedia Petri Net
Sound
Bm p
Text
V ideo
U ser m essage
W ave
U ser m essage
M idi

An example for the Compute-Node-Time
algorithm
70
35
20
15
15
20
40
30
20
a
b
c
d e

Topological sort of the transition nodes
a
c
d
e
b
20
70
152030
35
15
40
20

A Multimedia Presentation
Example
sound
text
musicanimation
video
image
a
b
c
d

Presentation Schedule of the Example
A nim ation
Sound
Text
M usic
Im age
V ideo
Tim e
Transition "a" Transition "b" Transition "c" Transition "d"
U ser transition

Synchronous Sets for the Order of
Transitions Firing
a
b
c
d
b c
d
C ase1 : W ithout firing of user transition
: {anim ation,sound,text}
: {m usic,im age,text}
: {m usic,video}
: End
C ase2 : W ith firing of user transition
: {m usic,im age,video}
: End

Layered Multimedia Presentation
Subnet
Subnet
Subnet
Transition Transition
U ser Transition

Implementation Example (cont’)

Summary
• Provide a model for the multimedia presentation
design by extending concepts of Petri Net
• Could be extended for layered multimedia
presentations
• Petri Net is used as an example of synchronization
control

Collaborative Learning with
Petri Net
• Different students learn different contents
and in different speed
• Team members have to learn skill and
gathered together to solve a mission
start
C
B
A
Dispatch Discuss Grade

The Same Structure but
Different Skill
Learner A
Cooperative Learning
Learner B
Learner C
L2L1
L3 L4
L1,L3,L4
L1,L2,L3,L4
L1,L2,L3,L4

Capability & Assessment
Learner A
Learner B
Learner C
L1,L3,L4
L1,L2,L3,L4
Exam
Give different
color tokens
condition
Learner D
L1,L2,L3,L4
L1,L2,L3,L4
Learner E
L1,L3,L4
C.L.

Cooperative Learning
Environment
Enter
Communication
Vote
Suspend
ExitExam

Interval Temporal Logic
• Temporal Intervals
• Fast Interval Relation Composition
• Solving the Composition Conflicts
• Interval Relation Distances
• Qualitative Spatial Knowledge
• The Applications

• James F. Allen, "Maintaining Knowledge about
Temporal Intervals," Communications of the ACM, Vol.
26, No. 11, 1983.
• Relations between two time intervals
• Relation composition
• Can be used in many research areas
Temporal Intervals

Allen’s 13 Temporal
Interval Relations

Composition of the 13
Interval Relations

From Point Relations to
Interval Relations

Revised 18 Temporal
Interval Relations

Fast Interval Relation
Composition
• Relation composition involves set operations
• The amount of relation sets is finite
• Table13

The Relation Composition
Algorithm
Algorithm : RelComp
Input : rs1 29RelSet, rs2 29RelSet
Output : rs 29RelSet
Preconditions : true
Postconditions : true
Steps :
1. rs = r1 rs1 , r2 rs2 (r1 , r2) rs1 rs2
Table13 (r1 , r2)

Computing All Possible
Relations
Algorithm : ComputeTable29
Input : Table13
Output : Table29
Postconditions : relation composition is closed under I
Steps:
1. Construct a set of 13 atomic sets from the 13 relations, assuming that this set is called I,
which is an index set for table look up.
2. Let Table29( i, j ) = Table13( i, j ), i I, j I
3. Table29( i, j ), i I , j I , do
3.1: if k = Table29( i, j ) I then
3.1.1 : I = I Table29( i, j )
3.1.2 : m I ,do
3.1.2.1 Table29( k, m ) = Relcomp( k, m )
3.1.2.2 Table29( m, k ) = Relcomp(m, k )

The 29 Possible Relation
Sets
IDs Relation Sets IDs Relation Sets
------------------------------------------------------------------------------------------
1 { < } 16 { o, d ,s }
2 { > } 17 { oi, di, si }
3 { d } 18 { <, o, m }
4 { di } 19 { >, oi, mi }
5 { o } 20 { f, fi, e }*
6 { oi } 21 { s, si, e }*
7 { m } 22 { <, o, m, d, s }
8 { mi } 23 { >, oi, mi, di, si }
9 { s } 24 { <, o, m, di, fi }
10 { si } 25 { > , oi, mi, d, f }
11 { f } 26 { o, oi, d, di, s, si, f, fi, e }*
12 { fi } 27 { <, m, d, di, o, oi, f, fi, s, si, e }
13 { e }* 28 { > , mi, di , d, oi, o, fi, f , si, s, e }
14 { o, di, fi } 29 { < , > ,m, mi, di , d, oi, o, fi, f , si, s, e }*
15 { oi, d, f }
rs rs-1 rs rs-1
-------------------------------------------
1 2 18 19
3 4 20 20
5 6 21 21
7 8 22 23
9 10 24 25
11 12 26 26
13 13 27 28
14 15 29 29
16 17
The Inverse Relations

Table29
o | 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
----------------------------------------------------------------------------------------------------------------------------- --------------
01 |01 29 22 01 01 02 01 22 01 01 22 01 01 01 22 22 22 01 29 22 01 22 29 01 29 22 22 29 29
02 |29 02 25 02 25 02 25 02 25 02 02 02 02 25 25 25 02 29 02 02 25 29 02 29 25 25 29 25 29
03 |01 02 03 29 22 25 01 02 03 25 03 22 03 29 25 22 29 22 25 22 25 22 29 29 25 29 29 29 29
04 |24 23 26 04 14 17 14 17 14 04 17 04 04 14 26 26 17 24 23 17 14 27 23 24 28 26 27 28 29
05 |01 23 16 24 18 26 01 17 05 14 16 18 05 24 26 22 27 18 28 22 14 22 29 24 28 27 27 29 29
06 |24 02 15 23 26 19 14 02 15 19 06 17 06 28 25 26 23 27 19 17 25 27 23 29 25 28 29 28 29
07 |01 23 16 01 01 16 01 20 07 07 16 01 07 01 16 22 22 01 28 22 07 22 29 01 28 22 22 29 29
08 |24 02 15 02 15 02 21 02 15 02 08 08 08 25 25 15 02 27 02 08 25 27 02 29 25 25 29 25 29
09 |01 02 03 24 18 15 01 08 09 21 03 18 09 24 15 22 27 18 25 22 21 22 29 24 25 27 27 29 29
10 |24 02 15 04 14 06 14 08 21 10 06 04 10 14 15 26 17 24 19 17 21 27 23 24 25 26 27 28 29
11 |01 02 03 23 16 19 07 02 03 19 11 20 11 28 25 16 23 22 19 20 25 22 23 29 25 28 29 28 29
12 |01 23 16 04 05 17 07 17 05 04 20 12 12 14 26 16 17 18 23 20 14 22 23 24 28 26 27 28 29
13 |01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
14 |24 23 26 24 24 26 24 17 14 14 26 24 14 24 26 27 27 24 28 27 14 27 29 24 28 27 27 29 29
15 |24 02 15 29 27 25 24 02 15 25 15 27 15 29 25 27 29 27 25 27 25 27 29 29 25 29 29 29 29
16 |01 23 16 29 22 28 01 23 16 28 16 22 16 29 28 22 29 22 28 22 28 22 29 29 28 29 29 29 29
17 |24 23 26 23 26 23 14 23 26 23 17 17 17 28 28 26 23 27 23 17 28 27 23 29 28 28 29 28 29
18 |01 29 22 24 18 27 01 27 18 24 22 18 18 24 27 22 27 18 29 22 24 22 29 24 29 27 27 29 29
19 |29 02 25 23 28 19 28 02 25 19 19 23 19 28 25 28 23 29 19 23 25 29 23 29 25 28 29 28 29
20 |01 23 16 23 16 23 07 23 16 23 20 20 20 28 28 16 23 22 23 20 28 22 23 29 28 28 29 28 29
21 |24 02 15 24 24 15 24 08 21 21 15 24 21 24 15 27 27 24 25 27 21 27 29 24 25 27 27 29 29
22 |01 29 22 29 22 29 01 29 22 29 22 22 22 29 29 22 29 22 29 22 29 22 29 29 29 29 29 29 29
23 |29 23 28 23 28 23 28 23 28 23 23 23 23 28 28 28 23 29 23 23 28 29 23 29 28 28 29 28 29
24 |24 29 27 24 24 27 24 27 24 24 27 24 24 24 27 27 27 24 29 27 24 27 29 24 29 27 27 29 29
25 |29 02 25 29 29 25 29 02 25 25 25 29 25 29 25 29 29 29 25 29 25 29 29 29 25 29 29 29 29
26 |24 23 26 29 27 28 24 23 26 28 26 27 26 29 28 27 29 27 28 27 28 27 29 29 28 29 29 29 29
27 |24 29 27 29 27 29 24 29 27 29 27 27 27 29 29 27 29 27 29 27 29 27 29 29 29 29 29 29 29
28 |29 23 28 29 29 28 29 23 28 28 28 29 28 29 28 29 29 29 28 29 28 29 29 29 28 29 29 29 29
29 |29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29

Solving the Composition
Conflicts
• Unknown derivation
If X < Y and Y > Z, then X ? Z
• Multiple derivation
If X < Y and Y d Z, then X {<, o, m, d, s} Z
• Conflict derivation
If X < Y, Y < Z, and X > Z are declared by the user,
then there exists a conflict

The Domain of Relations
• An user edge denotes a relation between a pair of objects defined by
the user. The relation may be reasonable or non-reasonable.
• A derived edge holds a non-empty set of reasonable relations derived
by our algorithm. The relation of the two objects connected by the
derived edge can be any reasonable relation in the set.
• complete relation domain (a complete graph): contains user edges
and derived edges, with possible cycles and possible conflicts.
• reasonable relation domain (a graph): contains user edges and
derived edges, with possible cycles but no conflict.
• reduced relation domain (a graph): contains only user edges, with
possible cycles and possible conflicts.
• restricted relation domain (a tree): contains only user edges, without
cycle.

Computes the Reasonable
RD from the Reduced RD
Algorithm : ComputeRD1
Input : G = ( GV,GE )
Output: Kn = ( KnV, KnE )
Postconditions : GV = KnV GE KnE
Steps :
1 : G = EliminateConflicts (G)
2 : Kn = G pl = 2
3 : repeat until | KnE | = | KnV | * ( | KnV | -1 ) / 2
3.1 : for each e = ( a, b ) e KnE a KnV b KnV such that
there is a path of user edges from a to b , with path length = pl
3.2 : suppose (( n1, n2 ), ( n2, n3 ),…, ( nk-1, nk ) )
is a path with a = n1 b = nk k = pl + 1
3.3 : set e.rs = Table29 (( a, nk-1 ).rs, (nk-1, b ).rs )
3.4 : KnE = KnE { e }
3.5 : pl = pl + 1

Eliminate Conflicts
Algorithm : EliminateConflicts
Input : G = ( GV, GE )
Output : G = ( G V, G E)
Preconditions : G contains only user edges G = G
Postconditions : G = G , but the reasonable sets of edges in G may be
changed.
Steps :
1. for each P = ((n1, n2), (n2, n3), , (nk-1, nk) ) in G with n1 = nk k >3
1.1 : for each i, 1 i k-2
1.1.1 : set (ni, ni+2).rs = Table29 ((ni, ni+1).rs, (ni+1, ni+2).rs )
1.2 : rs = Table29 ((nk, nk-2).rs, (nk-2, nk-1).rs )
1.3 : if (nk, nk-1).r rs then
1.3.1 : ask user to choose a r rs
1.3.2 : set (nk, nk-1).r = r

User Edges and Derived
Edges
User edges:
( A, B ) = { < } = [1]
( B, C ) = { m } = [7]
( C, D ) = { d } = [3]
( C, E ) = { s } = [9]
( F, D ) = { < } = [1]
2. Path Length = 3
(A, E) = (A, B ) o (B, C) o (C, E) = (A, C ) o (C, E)
= [1] o [9] = [1] = { < }
(A, D) = (A, B ) o (B, C) o (C, D) = (A, C ) o (C, D)
= [1] o [3] = [22] = { <, o, m, d, s }
(B, F) = (B, C ) o (C, D) o (D, F) = (B, D ) o (D, F)
= [16] o [1]-1 = [23] = { >, oi, mi, di, si }
(E, F) = (E, C ) o (C, D) o (D, F) = (E, D ) o (D, F)
=[14] -1 o [2] = [15] o [2] = [2] = { > }
3. Path Length = 4
(A, F) = (A, B ) o (B, C) o (C, D ) o (D, F)
= ( (A, B ) o (B, C) ) o ( (C, D ) o (D, F) )
= (A, C ) o (C, F) = [1] o [2] = [29]
= { <, >, d, di, o, oi, m, mi, f, fi, s, si, e }
Derived edges:
1. Path Length = 2
(A, C) = (A, B ) o (B, C) = [1] o [7] = [1] = { < }
(B, D) = (B, C ) o (C, D) = [7] o [3] = [16] = {o, d, s }
(C, F) = (C, D ) o (D, F) = [3] o [1]-1 = [3] o [2] = { > }
(D, E) = (D, C ) o (C, E) = [4] o [9] = [14] = {o, di, fi }
(B, E) = (B, C ) o (C, E) = [7] o [9] = [7] = { m }

Adds User Edges to the
Reasonable RD
Algorithm : AddUERD
Input : l = (a, b ), Kn = ( KnV, KnE )
Output : Kn+1 = ( Kn+1V, Kn+1E )
Preconditions : l KnE a KnV b KnV
Postconditions : | Kn+1V | = | KnV | + 1
| Kn+1E | = | KnE | + n
Steps :
1: Kn+1E = KnE { l }
2: for each e = ( c, b ) c a c KnV
2.1: e.rs = d KnV, ( c, d ) KnE, (d, b ) KnE
( Table29(( c, d ).rs, ( d, b ).rs ))
2.2: Kn+1E = Kn+1E {e}
3: Kn+1V = KnV { b }

Adding Edges
Adding an user edge to Example 4.1:
Add ( G, E ) = { f } = [11]
Derived edges :
Derive ( E, G ) = ( G, E )-1 = [11]-1 = [12] = { fi } ( inverse )
Derive ( A, G ) = ( A, E ) o ( E, G ) = [1] o [12] = [1] = { < } ( association )
Derive ( B, G ) = ( B, E ) o ( E, G ) = [7] o [12] = [1] = { < } ( association )
Derive ( C, G ) = ( C, E ) o ( E, G ) = [9] o [12] = [18] = {<, o, m }
Derive ( D, G ) = ( D, E ) o ( E, G ) = [14] o [12] = [24] = { <, o, m, di, fi } ( association )
Derive ( F, G ) = ( F, E ) o ( E, G ) = ( E, F )-1 o ( E, G ) = [1] o [12] = [1] = { < } ( inverse, association )

Interval Relation Distances
• Use relation distances to estimate the
similarity of object positions
• There are three types of relation
distances
– The Point Relation Distance
– The Interval Relation Distance
– The Extended Point-Interval Relation
Distance

The Interval Relation Distance

The Extended Point-Interval
Relation Distance

The Distance Graph
<
ols
m
loe
o
s
d
fi
e
f
di
si
oi m i
ole
los
>
oo

Qualitative Spatial
Knowledge
• Relation Embedding
– two points on a line
– two points on a plan
– two line segments on
a line
– two line segments on
a plan
– two 2-D objects on a
plan

Projection of Two Line
Segments on a 2-D Plan

Projection of Two 2-D
Objects on a 2-D Plan

Generalized Spatial Relation
f 1 = 29RelSet o 29RelSet 29RelSet
f 2 = 29RelSet o 29RelSet 29RelSet o 29RelSet 29RelSet 29RelSet
f 3 = 29RelSet o 29RelSet 29RelSet o 29RelSet 29RelSet o 29RelSet
29RelSet 29RelSet 29RelSet
where 29RelSet 29RelSet { {<} {<}, {<} {>}, … , { = } { = } }
29RelSet 29RelSet 29RelSet { {<} {<} {<}, {<} {<} {>},…, {=} {=} {=}}
i1 j1, i2 j2 P (29RelSet 29RelSet )
f 2 (i1 j1, i2 j2 ) = f 1 (i1 , i2 ) f 1 ( j1 , j2 )
i1 j1 k1 , i2 j2 k2 P (29RelSet 29RelSet 29RelSet )
f 3 (i1 j1 k1, i2 j2 k2) = f 1 (i1 , i2 ) f 1 ( j1 , j2 ) f 1 (k1 , k2 )
where A B = { a b | a A, b B }
A B C = { a b c | a A, b B, c C }

The Applications
• Multimedia Presentation Generation
• Shape-Based Image Retrieval (not
discussed)

Generation

Notes
• Synchronization of Multimedia Objects
• Computation Efficiency
• Extension toward N-Dimension
• Multimedia Presentation Scheduling

Summary
• White Board and Chat Room
• Multimedia Synchronization
• SMIL
• Petri Net
• Interval Temporal Logic

03 synchronized distance learning

Recomendados

Recomendados

Más contenido relacionado

Similar a 03 synchronized distance learning

Similar a 03 synchronized distance learning (20)

Más de 宥均林

Más de 宥均林 (20)

Último

Último (20)