IDEAS 2013 Presentation

 Copyright 2010 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.deri.ie
On-The-Fly Generation of
Multidimensional Data Cubes for
Web of Things
Muntazir Mehdi
(DERI, TU Kaiserslautern)
Stefan.Decker@deri.org
http://www.StefanDecker.org/

Agenda
 Motivation and Background
• Problem statement, Use case, Linked Data, WoT
 Processing Metadata for Cube Creation
• Capturing and Publishing Sensor data, Event Registration
 Cube Generation
• EDWH Agent, An example Scenario
 Other Potential Use Cases
 Results and Evaluation
 Conclusion

Motivation & Background
 Enterprises producing huge amounts of data
making data management, exchange and decision
making complex.
 Use Case (Smart Buildings)
1. Rely on Sensor data for decision making
2. Heterogeneous and Big Data Management
3. Event Processing can be applied to sustain decision making
4. Limited support for decision making with event processing
techniques
5. Controlling supply / demand based on statistical data
6. Identify meaningful event and deal with them asap

Motivation & Background (continued)
 Heterogeneous Data Management
1. Different Data generated from different applications within one or
more smart environments.
2. For example: A smart city relying on combined data from different
smart buildings.
3. Linked data: A set of best practices to represent data into RDF and
link, relate or connect to other RDF data.
4. Linked Open Data (LOD) Cloud: A huge openly available cloud of
linked data from different domains.

 Big Data Management
1. A fast response to complex queries to support event processing.
2. Huge amounts of sensor data as RDF.
3. Generation of real-time multidimensional and contextual data cubes
to sustain fast responses to complex queries.
4. An event data-warehouse.
5. Multidimensional shape of data in data-warehouse = A data cube =
Structuring information into dimensions and facts or measures.

 Why Data-warehouse for events?
1. Data characteristics:
• Logged once, never updated
• Flat data, no need to normalize
• Incoming data: temporal (based on time)
2. Objective characteristics:
• Reporting, Analysis, Prediction, Mining, Pattern Identification……
• To use a data model to speed up querying unlike transactional processing system
• To provide with a historical repository containing features as per interest
• Support Complex Event Processing

 Web of Things
1. Extending the Web to easily blend real-world objects like electronic
appliances, sensors and embedded devices etc.
2. Even though we are limited to sensor data in our use case, the
approach can be easily extended.
3. CoAP (Constrained Application Protocol): A Web transfer protocol for
request/response model.

Related Work
 Antoniades, Athos, et al. "Linked2Safety: A secure linked
data medical information space for semantically-
interconnecting EHRs advancing patients' safety in medical
research." Bioinformatics & Bioengineering (BIBE), 2012 IEEE
12th International Conference on. IEEE, 2012.
 Lefort, Laurent, et al. "A Linked Sensor Data Cube for a 100
Year Homogenised Daily Temperature Dataset." SSN. 2012.
 ENERGIE VISIBLE
(http://www.webofthings.org/energievisible/)

Processing Metadata for Cube Generation
Involves two major steps:
1. Capturing and Publishing Sensor Data
2. Event Registration

Capturing and Publishing Sensor Data: An
example Scenario
JMS
SERVER
& publish on JMS Server
RDF
Oh wait,
I see a way of converting them into RDF,
add relevant metadata,
SSN
Event Stream
Event Stream
Event Stream

Capturing and Publishing Sensor Data:
Process
Filter
UDP Listeners
&
CoAP Clients
RDFizer
JMS Publisher Enricher
JMS Server Metadata
Knowledge Base
S1
S2
S3
Sn

Event Registration: EDWH Ontology
NamedCubeGraph
Configuration
Dimension
Measure
Source Event
JMSSource

Event Registration Process
Specify Event
Type
Specify Event
Source
Select
Measures
Select
Dimensions
Specify Graph
Details
EDWH Ontology Instance

Dimension Selection: Example

Measure Selection: Example

Cube Generation
1. Requires an event to be registered into the system.
2. Current implementation generates cubes based on
time dimension only. However, it can be easily
extended to attain other dimensions.
3. Critical component: EDWH Agent

Cube Generation: EDWH Agent Architecture

Cubes Generation: An example Scenario
JMS
SERVER
RDF
RDF
CUBES AS
RDF MEETS Mr. CUBES
EDWH Ontology
CUBE
Store

Cubes Generation: Our Use Case

Use Cases
CUBE
Store

Use Case: 1
The electricity usage at location X for duration Y for consumer Z
has been moderate as compared to previous duration W.
CUBE
Store

Use Case: 2
Historical Data suggests that the weather is going to be windy and
Rainy in Galway even after the Easter.
CUBE
Store

Use Case: 3
CUBE
Store
Some suspicious activity has been detected on your credit card!

Use Case: 4
Linked
CUBE
Stores
Each of these things
can be achieved from
one place

Evaluation
 We evaluated our system in terms of
1. Total number of cubes generated
2. Size of each cube
3. Accuracy of generated cubes
4. Impact of adding and removing dimensions on size of cube
5. Performance of the system to generate cubes
6. Query Execution Time (QET)

Evaluation
0
2000
4000
6000
8000
10000
12000
14000
16000
Time(milliseconds)
Quarter Cube
Day Cube
Hour Cube

Evaluation: Size

Evaluation: Impact of dimensions
1 Dim
1 Dim
1 Dim
2 Dim
2 Dim
2 Dim
3 Dim
3 Dim
3 Dim
0
50
100
150
200
Quarter Hour Day
StorgaeSizeperCube(KB)
1 Dim 2 Dim 3 Dim

Evaluation: Query Set for QET

Evaluation: QET Comparison

Conclusion
With the approach presented, we were able to enrich
events with necessary metadata, and process
enriched events to generate on-the-fly data cubes.
After looking at performance chart shown in previous
slides, it is safe to conclude that our approach
provides a good way of generating data cubes on-the-
fly in a real-time sensor network.

Questions

IDEAS 2013 Presentation

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (8)

Similar a IDEAS 2013 Presentation

Similar a IDEAS 2013 Presentation (20)

Último

Último (20)

IDEAS 2013 Presentation