About the Workshop: The Stream Reasoning Workshop took place from January 16th to 17th, 2018. Processing, querying and reasoning over streaming data is studied in different communities such as KR&R, Semantic Web, Databases, Stream Processing, Complex Event Processing, etc., where researchers have different perspectives and face different challenges. This workshop aims at advancing Stream Reasoning as research theme by bringing together these different views and goals. In addition to invited talks, the workshop will provide opportunities for all participants to engage in discussions on open problems and future directions. (http://www.ifi.uzh.ch/en/ddis/events/streamreasoning2018.html) About the Talk: Real-time sensor data enables diverse applications such as smart metering, traffic monitoring, and sport analysis. In the Internet of Things, billions of sensor nodes form a sensor cloud and offer data streams to analysis systems. However, it is impossible to transfer all available data with maximal frequencies to all applications. Therefore, we need to tailor data streams to the demand of applications. We contribute a technique that optimizes communication costs while maintaining the desired accuracy. Our technique schedules reads across huge amounts of sensors based on the data-demands of a huge amount of concurrent queries. We introduce user-defined sampling functions that define the data-demand of queries and facilitate various adaptive sampling techniques, which decrease the amount of transferred data. Moreover, we share sensor reads and data transfers among queries. Our experiments with real-world data show that our approach saves up to 87% in data transmissions.

1. 1 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Optimized On-Demand Data Streaming from Sensor Nodes
Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl
Stream Reasoning Workshop 2018

2. 2 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
The Internet of Things
Real-time
insights
IoT

3. 3 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
The Internet of Things
Real-time
insights
Billions of sensor nodes form the IoT
and provide data streams to analysis systems.
IoT

4. 4 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
The Internet of Things
Real-time
insights
Billions of sensor nodes form the IoT
and provide data streams to analysis systems.
IoT

5. 5 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
The Internet of Things
Real-time
insights
Billions of sensor nodes form the IoT
and provide data streams to analysis systems.
IoT

6. 6 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
The Internet of Things
Real-time
insights
Billions of sensor nodes form the IoT
and provide data streams to analysis systems.
IoT

7. 7 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Problem

8. 8 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Problem
Heavy Network Utilization

9. 9 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Problem
Heavy Network Utilization Scalability Challenges

10. 10 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Problem
Heavy Network Utilization Scalability Challenges Increasing Latencies

11. 11 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Problem
Heavy Network Utilization Scalability Challenges Increasing Latencies
Financial Costs

12. 12 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Solution Outline
Tailor Data Streams to the Demand of Applications

13. 13 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Solution Outline
Tailor Data Streams to the Demand of Applications
• Provide an abstraction to define the data demand of applications.
User-Defined Sampling Functions (UDSFs)

14. 14 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Solution Outline
Tailor Data Streams to the Demand of Applications
• Provide an abstraction to define the data demand of applications.
• Optimize communication costs while maintaining the result accuracy.
User-Defined Sampling Functions (UDSFs)
Read-Time Optimization

15. 15 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Solution Outline
Tailor Data Streams to the Demand of Applications
• Provide an abstraction to define the data demand of applications.
• Optimize communication costs while maintaining the result accuracy.
• Share sensor reads and data transfer among users and queries.
User-Defined Sampling Functions (UDSFs)
Read-Time Optimization
Multi-Query / Multi-User Optimization

16. 16 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Architecture Overview

17. 17 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Architecture Overview

18. 18 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Architecture Overview

19. 19 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Architecture Overview

20. 20 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Architecture Overview

21. 21 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Sensor Read Scheduling

22. 22 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
User Defined Sampling Functions
Input:
Sensor read time and value
Output:
Next Sensor Read Request

23. 23 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
User Defined Sampling Functions
Enable adaptive sampling techniques to reduce data transmission
e.g., Adam [Trihinas ‘15], FAST [Fan ‘14], L-SIP [Gaura ’13]

24. 24 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
User Defined Sampling Functions
Enable model-driven data acquisition

25. 25 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
User Defined Sampling Functions
Enable model-driven data acquisition

26. 26 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Read Fusion and Read Time Optimization

27. 27 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Read Fusion and Read Time Optimization
1) Minimize Sensor Reads and Data Transfer

28. 28 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Read Fusion and Read Time Optimization
1) Minimize Sensor Reads and Data Transfer
2) Optimize Sensor Read Times:
● Check the paper for all details on the read time optimizer!

29. 29 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17

30. 30 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Local Filtering
Enable model-driven data acquisition / filtering
Enable adaptive filtering algorithms
…or simply filter push-down

31. 31 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Experiments
• Replay formula 1 telementry data (speed, rpm, acceleration, …)
• Random UDSFs:
- Read in a poisson process (also simulate load peaks)
- In average 1 read per query per second
- Exponentially distributed read time tolerance
- high probability for small tolerances
- small probability for large tolerances
- In average 0.04s read time tolerance

32. 32 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Shared Sensors: Concurrent UDSFs Experiment
• On-Demand scheduling reduces sensor reads and data transfer by up to 87%.
• The # of reads and transfers increases sub-linearly with the # of queries.

33. 33 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Shared Sensors: Read Time Optimization
• Our read-time optimizer reduces the deviation from desired read times
by up to 69% (preserving the min. # of reads and transfers).

34. 34 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Impact of Read-Time Tolerances

35. 35 © BBDC
Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
Optimized On-Demand Data
Streaming from Sensor Nodes
Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl
ACM Symposium on Cloud Computing (SoCC) 2017
Wrap-Up:
Tailor Data Streams to the Demand of Applications
• Define data demand: User-Defined Sampling Functions
• Schedule sensor reads and data transfer on-demand
• Optimize read times globally - for all users and queries