2. • Why am I excited about things?
• What are We doing?
• What are the challenges?
3. What is your research interest?
• Computer Vision
• Multimedia
• Machine Learning
• Social Media
• Intelligent Systems
• Big Data
• Stream Processing
Building a Better Society.
4. Events and Entities Exist in the real
world.
Events and entities result in Data and
Documents
5. What is the most Fundamental
Problem in Society?
Connecting People to Resources
Effectively, Efficiently, and Promptly
in given Situations.
Hint: Economics, Health Care, Politics, Computer Science,
Operations Research, …
18. Motivation
Location Based
Mobile Applications
Ongoing Archived
Database System
satellite
Environmental
Sensor Devices
Internet of Things
Social Media
Social Life
Network
Experts
People
Governmental
Agencies
Situations
21. • Social observations are now possible with
little latency.
• We can design social systems with feedback.
• Situation Recognition and Need-Availability
identification of resources is a major
challenge.
22. Smart Social Systems Architecture
Situation
Recognition
Evolving
Situations
Available
Resources
Identified
Needs
Need-
Resource
Matcher
Communicati
on/Control
Unit
Resources
People
D
a
t
a
S
o
u
r
c
e
s
Database System
satellite
Environmental
Sensor Devices
Social Media
23. Concept Recognition: Last Century
23
Environm
ents
Real world
Objects
Situations
Activities
SingleMedia
SPACE
TIME
ScenesLocation
aware
Visual
Objects
Trajectories
Visual
Events
Location
unaware
Static Dynamic
Location
aware
Location
unaware
Static Dynamic
Data = Text or Images or Video
25. Concept Recognition: This Century
25
Environm
ents
Real world
Objects
Situations
Activities
SPACE
TIME
Location
aware
Location
unaware
Static Dynamic
HeterogeneousMedia
Location
aware
Location
unaware
Static Dynamic
Data is just Data.
Medium and sources do not matter.
26. • relative position or combination of
circumstances at a certain moment.
• The combination of circumstances at a
given moment; a state of affairs.
29. • Example 1:
– A person shouting.
– 1000 people shouting.
• In a contained building
• In main parts of a city
• Example 2:
– One person complaining about flu.
– Many people from different areas of a country
complaining about flu.
30. Micro-events:
Sensors detecting and chirping
(broadcasting) events
• Billions of disparate kinds of sensors being
placed everywhere.
• Each sensor detects ‘basic events’ and
broadcasts it in a simple form.
• Develop a system to process these micro-
events and make them useful.
31. Example: Cameras in a city
• ‘Chirps’ could be of different types
• Define behaviors like:
– Heavy traffic
– Popular event going on
– People leaving X area
– Violence starting
– . . .
• Use for Macro-behvior analysis
32. From micro events
to situations
Thermodynamics
provides a framework for relating the
microscopic properties of individual
atoms and molecules to the
macroscopic or bulk properties of
materials that can be observed in
everyday life.
33. Data Types in Situation Recognition
• Static Data:
– POIs (location of hospitals)
– Population
– Technical data of hospitals
– Contact persons in committees or health organization, and
– All information on support-potentials for personnel, material and
infrastructure
• Dynamic Data:
– Current disease data
– Twitter/Google Trends
– Environmental data
– Personal Individual data
34. Two Big Challenges
• Data Ingestion to efficiently extract data from
the Web and make them available for later
computation is not-trivial.
• Stream Processing Engine to bridge the
semantic gap between high level concept of
situations and low level data streams.
35. S
Social
Networks
2-D spatial
Grid at time
T
Database
Systems
Global
Sensors
Phone
Apps Internet of
Things
(S, T, T)
S Uses
Application
semantics to
combine
different data
items.
45. 8/21/2013 45
Billions of data sources.
Environment for
Selecting, and
Combining
appropriate sources to detect situations.
Prediction for Pro-active actions
Interactions with different types of Users
Decision Makers
Individuals
46. OutputIngestor
EventSource
Parser
Data Adapter
Emage
Generator
(+resolution mapper)
Processing
EvShop Internal Storage
Query
Parser
Query
Rewriter
Event Stream Processing
Executor
ᴨ
ᴨ
Action Parser
Register EventSource Register Continuous Query
Situation
Emage
Visualization
(Dashboard)
Actuator
Communication
Action Control
Event Property &
Other Information
(e.g., spatio-temporal
pattern)
µ
Data Access Manager
Live Stream
Archived Stream
Situation Stream
Real-Time
DataSource
(e.g., sensor
streams, geo-image
streams)
Near Real-Time
DataSource
(e.g., preprocessing
data streams, social
media streams)
Raw Event
47. Input Manager
External Event
Preprocessing
(Collaboration)
47
Real-Time Sensor Streams
e.g., Cloud Satellite Images
Real-Time Sensor Streams
e.g., Wind Speed, Traffic Flow
RealTime DataSource
1D Data
Wrapper
STT to Emage
2D Data
Wrapper
Data Adapter Emage Generator
Emage
Emage
Factory
STT
Emage
Raw Social Media Streams
e.g., Twitter, News RSS Feed
NearRealTime DataSource
Event Model
Wrapper
STT to Emage
Data Adapter Emage Generator
Emage
Emage
Factory
STT
Topic Event
Detection
Abnormal
Event
Detection
Raw Sensor Streams
e.g., PM2.5 data
“EventModel” Streams
e.g., suddenly change
of data trend
within time window
Emage Store
STT Store
Metadata Store
EventSource
Parser Interface
(Optional)
RealTime
Emage Streams
NearRealTime
Emage Streams
Processing
Manager
ES Descriptor
ES Control
(Start/Stop/
View ES)
Users Input
Automatic Data/Events Flow
InitialResolution
AggregationFunc
Metadata
Theme
AdapterType
SourceURL
TimeWindow
Parameters
48. 48
Stream Processing Engine
Operators Manager
Built-in
Operators
User-Defined
Operators
ᴨ
ᴨ
µ
Data
Access
ᴨ
ᴨ
µ
Data
Access
ᴨ
ᴨ
µ
Data
Access
Input Manager
(Accessing
External Data)
Event Stream Executor
Operators
Nodes
Internal Storage
(Accessing Internal
Data)
AsterixDB, SciDB,
MongoDB
Emage Store
STT Store
Metadata Store
Query Parser
Interface
Query
Descriptor
Query Control
(Start/Stop/ View)
Real-time/
near real-time
Emage Streams
Archived
Emage Streams Situation Streams
Emage
Interpolation
Function
Emage
Conversion
Final
Resolution
Parameter Operators
Operators
Store Parameters
Retrieve Parameters
Query
Rewriter
Execution Plan
59. Meeting with
development group
Yoga with
Jena
Meeting with
test group
Mina’s birthday party
9:30-11:00 12 - 1
4:00-
5:00
Transportation
walking
walking
Transportation
Transportation
Transportation
Home HomeWork Caspian
HavingBreakfast
Doingthedishes
Commuting
Commuting
Walking
Walking
Meeting
Meeting
working
Working
Exercise
Exercise
Working
Working
Working
Working
Meeting
Meeting
Walking
Commuting
Commuting
Commuting
Shopping
Shopping
Commuting
Partying
Partying
Partying
Partying
Partying
Commuting
Commuting
Commuting
Sleeping
Sleeping
WatchingTV
Cleaning
Personicle
Commute Walk Meet work Exer. work Meet Commute shop party Commute Sleep
GPSlocationtracking
Calendar
Activity
levelPersonicle
61. Research Challenges
• Situation Recognition
• Persona and Personal Context
• Chronicle Analytics and Visualization
• Massive Geo-Spatial Heterogeneous
Stream Processing
• Dynamic Need-Resource Optimization
62. Situation Recognition
• Next Frontier in Concept Recognition
• Heterogeneous Geo-spatial Dynamic Data
• Social data and IoT become a key element
• Application and domain semantics
• Model definitions
• High dimensionality
• Unification of data: Social-Cyber-Physical
63. Persona and Personal Context
• Not only Logs of Keyboard and Surfing.
• You Log and explore every thing.
– Entity resolution on TURBO
• Many new data processing and unification
challenges.
65. MicroBlogs and Twitter:
LIMITATIONS
• Very LOW Signal-to-Noise ratio: High
Noise-to-Signal ratio
• Focus on being broad platform.
• Difficult to extract SIGNAL.
• Information must be extracted from
limited text.
66. Solution: Tweeting Applications
• Develop focused Apps: Focused MicroBlogs
• Get all information from ‘motivated’ and
collaborative users.
• Help them solve their problem.
68. Chronicle Analytics
• Enterprise Warehouse were for late 20th
Century – Planetary Warehouses are defining
this century.
• Big data is important because it collects
everything that happens to build ‘Prediction
Machines’.
• Machine learning and visualization are the
key tools.
69. Massive Geo-Spatial
Heterogeneous Stream Processing
• Why does the DATA become so BIG?
• And it will keep getting BIGGER.
• We have to go beyond Batch Processing as
primary computing approach.
• Should be of great interest to Social Media
researchers.
70. Dynamic Need-Resource Optimization
• What are the fundamental problems in
Computer Science?
– Time and memory compexity
– Operating systems, networks, storage management,
algorithms, …
• What is the main concern in
– Economics?
– Healthcare?
– Politics?
– …
71. Live EventShop and Collaboration
• Live EventShop Demo
– http://auge.ics.uci.edu/eventshop/
• Current Collaborators & Plan
– Cyber-Physical Cloud Computing Project
• NICT, NIST
– SLN4MOP Project
• Sri Lanka Farmers; Prof. Ginige in Sydney leading
– Open Source EventShop by mid-year of 2013
• HCL
72. Thanks for your time and attention.
For questions: jain@ics.uci.edu