SlideShare una empresa de Scribd logo
1 de 20
Efficiently Maintaining Distributed Model-
 Based Views on Real-Time Data Streams




     Alexandru Arion, Hoyoung Jeung, Karl Aberer
                     EPFL, 2011
Data networks


Local: low power connected devices transmit to base stations.

Large scale: base stations transmit over large distances using existing
communication infrastructure.
Relevance


Large numbers of sensor networks are already being
interconnected and share huge amount of streaming data.

Example: SwissEx (http://www.swiss-experiment.ch)
Related work
S. Shah, et all., “An efficient and resilient approach to filtering and disseminating
streaming data,” in VLDB, 2003, pp. 57–68.
Y. Zhou, et all., “Disseminating streaming data in a dynamic environment: an
adaptive and cost-based approach,” The VLDB Journal, vol. 17, no. 6, pp. 1465–
1483, 2008.
D. J. Abadi, et all., “The design of the Borealis stream processing engine,” in
CIDR, 2005, pp. 277–289.
M. Balazinska, et all., “Load management and high availability in the medusa
distributed stream processing system,” in SIGMOD, 2004, pp. 929–930.
P. Pietzuch, et all., “Network-aware operator placement for stream-
processing systems,” in ICDE, 2006, p. 49.
The framework
Key features (1)


Feature 1: reduces communication costs (does not
require any data transfer of actual streams)


Feature 2: any type of queries can be processed (all
data required for query processing is available to
consumer nodes)
Key features (2)


Feature 3: any type of model can be employed
(serves any application)


Feature 4: systematic solution that can guarantee
user-specified accuracy requirements for model-
based views.
Algorithms (1)


Coded model update:
● predetermines parameter values
● encodes them with bitmaps
● updates models efficiently sending only bitmaps
Algorithms (2)


Coded inter-variable model:
● uses correlation information
● reduces data redundancy
Framework properties

Accuracy requirements solution:
 ● The producer node generates a model-driven value when a new raw
   reading is streamed, and checks whether the difference between the
   raw value and the model-driven value stays within the error bound.
 


    ● If the difference does not exceed the error bound, no communication is
      required between the two nodes, and the consumer node generates
      values for their model-based views.
 


    ● Otherwise, the producer node reconstructs its model, so that the
      model-driven value generated from the reconstructed model does not
      exceed the error bound from the current raw reading. Next, the
      producer node updates the models at consumer nodes by sending
      new parameter values of the reconstructed model.
Coded Model Update
Coded Inter-variable Model
Coded Inter-variable Model (2)
Experiments (1)
Experiments (2)
Experiments (3)
Further related work
A. Deshpande and S. Madden, “MauveDB: supporting model-based user views in
database systems,” in SIGMOD, 2006
Y. Ahmad, O. Papaemmanouil, U. C¸ etintemel, and J. Rogers, “Simultaneous
equation systems for query processing on continuous-time data streams,” in ICDE,
2008
A. Thiagarajan and S. Madden, “Querying continuous functions in a database
system,” in SIGMOD, 2008
A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, "Compressing historical
information in sensor networks,” in SIGMOD, 2004
H. Chen, J. Li, and P. Mohapatra, “RACE: time series compression with
rate adaptivity and error bound for sensor networks,” 2004
S. Gandhi, S. Nath, S. Suri, and J. Liu, “Gamps: Compressing multi
sensor data by grouping and amplitude scaling,” in SIGMOD, 2009
Conclusions


● Generic framework

● Arbitrary numerical models

● Coded model update

● Coded inter-variable model

Más contenido relacionado

Destacado

Mystartingpointworkbook
MystartingpointworkbookMystartingpointworkbook
Mystartingpointworkbookkelseysadlerx
 
"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier OngenaCristal Events
 
Bittarget digital marketing-campaign
Bittarget digital marketing-campaignBittarget digital marketing-campaign
Bittarget digital marketing-campaignbittarget17
 
คำอธิบายรายวิชา
คำอธิบายรายวิชาคำอธิบายรายวิชา
คำอธิบายรายวิชาPrae Samart
 
Presentación1 examenfinal alicia perez
Presentación1 examenfinal alicia perezPresentación1 examenfinal alicia perez
Presentación1 examenfinal alicia perezAliciaPerezRuizDiaz
 
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyIntroduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyAndrey Golub
 
Security and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureSecurity and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureAchim Friedland
 
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationDr. Haxel Consult
 
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNDr. Haxel Consult
 
Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011benson215
 
API and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveAPI and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveCisco DevNet
 

Destacado (17)

Mystartingpointworkbook
MystartingpointworkbookMystartingpointworkbook
Mystartingpointworkbook
 
"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena
 
freeseminar
freeseminarfreeseminar
freeseminar
 
PlanetData Management Overview
PlanetData Management OverviewPlanetData Management Overview
PlanetData Management Overview
 
respostas 02
respostas 02respostas 02
respostas 02
 
Bittarget digital marketing-campaign
Bittarget digital marketing-campaignBittarget digital marketing-campaign
Bittarget digital marketing-campaign
 
คำอธิบายรายวิชา
คำอธิบายรายวิชาคำอธิบายรายวิชา
คำอธิบายรายวิชา
 
Presentación1 examenfinal alicia perez
Presentación1 examenfinal alicia perezPresentación1 examenfinal alicia perez
Presentación1 examenfinal alicia perez
 
Trabajo diseño final luis
Trabajo diseño final luisTrabajo diseño final luis
Trabajo diseño final luis
 
Week 6
Week 6Week 6
Week 6
 
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyIntroduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
 
Security and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureSecurity and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructure
 
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
 
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
 
Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011
 
Omron ladder programming
Omron ladder programmingOmron ladder programming
Omron ladder programming
 
API and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveAPI and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep dive
 

Similar a Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentSafayet Hossain
 
Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...ijtsrd
 
Evaluation of Different Machine.pptx
Evaluation of Different Machine.pptxEvaluation of Different Machine.pptx
Evaluation of Different Machine.pptxtariqqureshi33
 
Aplications for machine learning in IoT
Aplications for machine learning in IoTAplications for machine learning in IoT
Aplications for machine learning in IoTYashesh Shroff
 
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Luigi Vanfretti
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Otávio Carvalho
 
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesSensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesKarthik Sharma
 
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
Ncct   Ieee Software Abstract Collection Volume 1   50+ AbstNcct   Ieee Software Abstract Collection Volume 1   50+ Abst
Ncct Ieee Software Abstract Collection Volume 1 50+ Abstncct
 
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...IEEEGLOBALSOFTTECHNOLOGIES
 
Harvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksHarvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksIEEEFINALYEARPROJECTS
 
Reliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkReliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkIJMTST Journal
 
Energy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCEnergy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCpaperpublications3
 
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...IJERA Editor
 
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Jason Liu
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 tsysglobalsolutions
 

Similar a Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams (20)

Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud Environment
 
Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...
 
Evaluation of Different Machine.pptx
Evaluation of Different Machine.pptxEvaluation of Different Machine.pptx
Evaluation of Different Machine.pptx
 
Introduction_PPT.pptx
Introduction_PPT.pptxIntroduction_PPT.pptx
Introduction_PPT.pptx
 
Aplications for machine learning in IoT
Aplications for machine learning in IoTAplications for machine learning in IoT
Aplications for machine learning in IoT
 
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
 
Poster (1)
Poster (1)Poster (1)
Poster (1)
 
2. visualization in data mining
2. visualization in data mining2. visualization in data mining
2. visualization in data mining
 
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesSensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
 
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
Ncct   Ieee Software Abstract Collection Volume 1   50+ AbstNcct   Ieee Software Abstract Collection Volume 1   50+ Abst
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
 
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
 
Harvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksHarvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networks
 
cv_Md_Ariful_Islam
cv_Md_Ariful_Islamcv_Md_Ariful_Islam
cv_Md_Ariful_Islam
 
Reliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkReliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor Network
 
Big data analytics
Big data analyticsBig data analytics
Big data analytics
 
Energy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCEnergy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LC
 
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
 
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016
 

Más de PlanetData Network of Excellence

A Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoA Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoPlanetData Network of Excellence
 
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksOn Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksPlanetData Network of Excellence
 
Towards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingTowards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingPlanetData Network of Excellence
 
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamDemo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamPlanetData Network of Excellence
 
On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingPlanetData Network of Excellence
 
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...PlanetData Network of Excellence
 
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchLinking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchPlanetData Network of Excellence
 
SciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSSciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSPlanetData Network of Excellence
 
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduceScalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReducePlanetData Network of Excellence
 
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...PlanetData Network of Excellence
 
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsTowards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsPlanetData Network of Excellence
 
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...PlanetData Network of Excellence
 

Más de PlanetData Network of Excellence (20)

Dl2014 slides
Dl2014 slidesDl2014 slides
Dl2014 slides
 
A Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoA Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about Trentino
 
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksOn Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
 
Towards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingTowards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory Sensing
 
Privacy-Preserving Schema Reuse
Privacy-Preserving Schema ReusePrivacy-Preserving Schema Reuse
Privacy-Preserving Schema Reuse
 
Pay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching NetworksPay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching Networks
 
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamDemo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
 
On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream Processing
 
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
 
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchLinking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
 
SciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSSciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMS
 
CLODA: A Crowdsourced Linked Open Data Architecture
CLODA: A Crowdsourced Linked Open Data ArchitectureCLODA: A Crowdsourced Linked Open Data Architecture
CLODA: A Crowdsourced Linked Open Data Architecture
 
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduceScalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
 
Data and Knowledge Evolution
Data and Knowledge Evolution  Data and Knowledge Evolution
Data and Knowledge Evolution
 
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
 
Access Control for RDF graphs using Abstract Models
Access Control for RDF graphs using Abstract ModelsAccess Control for RDF graphs using Abstract Models
Access Control for RDF graphs using Abstract Models
 
Arrays in Databases, the next frontier?
Arrays in Databases, the next frontier?Arrays in Databases, the next frontier?
Arrays in Databases, the next frontier?
 
Abstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF DatasetsAbstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF Datasets
 
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsTowards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
 
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
 

Último

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditSkynet Technologies
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 

Último (20)

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 

Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

  • 1. Efficiently Maintaining Distributed Model- Based Views on Real-Time Data Streams Alexandru Arion, Hoyoung Jeung, Karl Aberer EPFL, 2011
  • 2. Data networks Local: low power connected devices transmit to base stations. Large scale: base stations transmit over large distances using existing communication infrastructure.
  • 3. Relevance Large numbers of sensor networks are already being interconnected and share huge amount of streaming data. Example: SwissEx (http://www.swiss-experiment.ch)
  • 4. Related work S. Shah, et all., “An efficient and resilient approach to filtering and disseminating streaming data,” in VLDB, 2003, pp. 57–68. Y. Zhou, et all., “Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach,” The VLDB Journal, vol. 17, no. 6, pp. 1465– 1483, 2008. D. J. Abadi, et all., “The design of the Borealis stream processing engine,” in CIDR, 2005, pp. 277–289. M. Balazinska, et all., “Load management and high availability in the medusa distributed stream processing system,” in SIGMOD, 2004, pp. 929–930. P. Pietzuch, et all., “Network-aware operator placement for stream- processing systems,” in ICDE, 2006, p. 49.
  • 6. Key features (1) Feature 1: reduces communication costs (does not require any data transfer of actual streams) Feature 2: any type of queries can be processed (all data required for query processing is available to consumer nodes)
  • 7. Key features (2) Feature 3: any type of model can be employed (serves any application) Feature 4: systematic solution that can guarantee user-specified accuracy requirements for model- based views.
  • 8. Algorithms (1) Coded model update: ● predetermines parameter values ● encodes them with bitmaps ● updates models efficiently sending only bitmaps
  • 9. Algorithms (2) Coded inter-variable model: ● uses correlation information ● reduces data redundancy
  • 10. Framework properties Accuracy requirements solution: ● The producer node generates a model-driven value when a new raw reading is streamed, and checks whether the difference between the raw value and the model-driven value stays within the error bound.
  • 11.   ● If the difference does not exceed the error bound, no communication is required between the two nodes, and the consumer node generates values for their model-based views.
  • 12.   ● Otherwise, the producer node reconstructs its model, so that the model-driven value generated from the reconstructed model does not exceed the error bound from the current raw reading. Next, the producer node updates the models at consumer nodes by sending new parameter values of the reconstructed model.
  • 19. Further related work A. Deshpande and S. Madden, “MauveDB: supporting model-based user views in database systems,” in SIGMOD, 2006 Y. Ahmad, O. Papaemmanouil, U. C¸ etintemel, and J. Rogers, “Simultaneous equation systems for query processing on continuous-time data streams,” in ICDE, 2008 A. Thiagarajan and S. Madden, “Querying continuous functions in a database system,” in SIGMOD, 2008 A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, "Compressing historical information in sensor networks,” in SIGMOD, 2004 H. Chen, J. Li, and P. Mohapatra, “RACE: time series compression with rate adaptivity and error bound for sensor networks,” 2004 S. Gandhi, S. Nath, S. Suri, and J. Liu, “Gamps: Compressing multi sensor data by grouping and amplitude scaling,” in SIGMOD, 2009
  • 20. Conclusions ● Generic framework ● Arbitrary numerical models ● Coded model update ● Coded inter-variable model