This document discusses using an event-web perspective and situation-based modeling to analyze multimodal event data from various sources and environments. It proposes connecting real users, events, and places rather than just documents. Key points include representing events and objects as basic linking mechanisms, allowing inference across events to derive meaningful situations, and applying this approach to applications like building control, telepresence, and emergency response to support decision making. The document outlines approaches for situation modeling and control using a situation calculus framework to reason about events and influence environments.

1. From Events to Situations: An
Event-web perspective
Vivek Singh
Advisor: Professor Ramesh Jain
University of
California, Irvine

2. Event-web
• Connecting real users, events and places rather
than just documents.
• Events and objects as basic organization and
linking mechanism
▫ Multimodal
▫ Closer to real world
• Users gain insights and experiences
• IBM Smarter planet

3. Event-web (imminent signs)
• Image and video sites for sharing experiential
data related to events
• Tweets about events of interest
• Multimodal news broadcast of events
• Detection of events in surveillance videos

4. Motivation: From events to situations…
• Given a plethora of event data. How can we:
▫ Disambiguate relevant and irrelevant events?
▫ Combine events into meaningful representations ?
▫ Allow inference and cascading effects
▫ Support different interpretations based on
application domain
▫ Support Control & decision making

5. Situation based control: Motivations
1. Inherent support for event-based (temporal)
reasoning
2. The ability of the controller to reason based on
symbols (rather than just signals)
3. Explicit inclusion of domain semantics (to
support multiple applications)

6. Applications
• Energy efficient buildings:
▫ When to switch off air-conditioner?
• Telepresence:
▫ Which camera feed to send out?
• Business analysis:
▫ What should be the correct price for iPhone?
• Earthquake rescue effort:
▫ Where to send out the next fire-fighter engine?

7. E2E communication: Project Overview
Environment 1 Environment 2
Device to Device
Sentient Sentient
Information
communication
Web Information
System System
Towards Environment to Environment (E2E) multimedia communication systems, in
Multimedia Tools and Applications Journal, Springer Netherlands, 2009.
Also in: ACM Workshop on Semantic Ambient Media Experiences (SAME), ACM
Multimedia workshop, 2008.

8. Environment: Node Architecture
EventBase
Sensors
Situation
Physical Environment Environment Network/
MMDB based
Environment Model Server Transmis
controller
sion
Actuators /
Presentation Actuator /
Devices Presentation
Model

9. Situation Calculus: Quick overview
▫ enter(P1), startWork(P1)
▫ enter(P1), exit(P1), enter(P1), startWork(P1), stopWork(
P1), startWork(P1)
- isInRoom(P1, s(k))
- isWorking(P1, s(k))
isInRoom(P1, s) 1
0
isWorking(P1, s) 1
0
isInRoom(P1, s) ˄~isWorking(P1, s) →
IncreaseMusicVolume()
Situation = Not events , nor sequence of events,
but their assimilated descriptor

10. Situation calculus
• Ω = {Actions, Situations, Objects, Fluents}
• Situation:
▫ “The set of necessary and sufficient world state
descriptors for undertaking control decision”.
• D = Dfnd U Duna U ε U Dap U Dss U D0
▫ Precondition axioms
▫ Successor-state axioms
▫ Initial situation
• Do(action, situation): A X S → S

11. Control theoretic problem formulation
• •
• •
• •

12. Situation modeling: E2E application
Loc 1: Desk Loc2: Whiteboard Conditions Actions
Move to Activity Selected Desired
location Cam Volume
Desk WorkOn 1 1
Actions possible: PC
1. Work on PC
2. Work on Table Desk WorkOn 2 2
Table
Whitebo - 3 3
ard
User Model - 4 4
Loc 3: Engineering
Model
Situation based control for cyber physical environments, Accepted: IEEE
workshop on situation management, MILCOM, 2009

13. Situ-itter: Large scale situations on
Twitter
• Looking beyond a room:
▫ Can an entire city or country
be considered a cyber physical system.
• Humans as sensors:
▫ Everywhere !
▫ Perception, Censors, Rumors, Delays
• Data has salient features:
 Unstructured, Noisy, Humungous, Spatial semantics
• Event detection is not well studied!

14. Situ-itter: First steps
• Spatio-temporal visualization for insights
• Spatio-temporal analysis for event detection
• Combining with external sources of information
for decision making
• Applications
▫ Event detection
▫ Should iPhone price be increased/decreased?
▫ Where and when to launch an ATT roadshow?

15. Comparison with external data
Aggregate interest on iPhone, Current ATT store location data

16. Where to have an ATT roadshow?
(using spatial-temporal convolution)
Location has semantics
<geoname>
<name>Sandy Big Bend Reservoir Number 1</name>
<lat>42.5191149</lat>
<lng>-109.4681887</lng>
<geonameId>5837570</geonameId>
<countryCode>US</countryCode>
<countryName>United States</countryName>
<fcl>H</fcl>
<fcode>RSV</fcode>
<fclName>stream, lake, ...</fclName>
<fcodeName>reservoir(s)</fcodeName>
<population>120,178<population/>
<alternateNames/>
<elevation>2194</elevation>
<continentCode>NA</continentCode>
<adminCode1>WY</adminCode1>
<adminName1>Wyoming</adminName1>
<adminCode2>035</adminCode2>
<adminName2>Sublette County</adminName2>
<timezone dstOffset="-6.0" gmtOffset="-
7.0">America/Denver</timezone>
<distance>3.3639</distance>
</geoname>

17. Future directions
• Tip of the iceberg:
▫ Spatio-temporal event detection in social media
• Reasoning/inference mechanisms
• Combining spatial, temporal and social
semantics into decision making
• Considering multi-modal data, user and sensor
based data
• A cyber-physical event-web which connects real
users and environments