Scalable and Reliable Multi-Dimensional Aggregation of Sensor Data Streams

Henning, Sören and Hasselbring, Wilhelm (2019) Scalable and Reliable Multi-Dimensional Aggregation of Sensor Data Streams [Paper] In: 2019 IEEE International Conference on Big Data (Big Data), 9-12 Dec. 2019, Los Angeles, CA, USA.

BigData2019StreamarXiv.pdf - Accepted Version

Download (380Kb) | Preview

Supplementary data:


Ever-increasing amounts of data and requirements to process them in real time lead to more and more analytics platforms and software systems being designed according to the concept of stream processing. A common area of application is the processing of continuous data streams from sensors, for example, IoT devices or performance monitoring tools. In addition to analyzing pure sensor data, analyses of data for groups of sensors often need to be performed as well. Therefore, data streams of the individual sensors have to be continuously aggregated to a data stream for a group. Motivated by a real-world application scenario, we propose that such a stream aggregation approach has to allow for aggregating sensors in hierarchical groups, support multiple such hierarchies in parallel, provide reconfiguration at runtime, and preserve the scalability and reliability qualities induced by applying stream processing techniques. We propose a stream processing architecture fulfilling these requirements, which can be integrated into existing big data architectures. We present a pilot implementation of such an extended architecture and show how it is used in industry. Furthermore, in experimental evaluations we show that our solution scales linearly with the amount of sensors and provides adequate reliability in the case of faults.

Document Type: Conference or Workshop Item (Paper)
Keywords: Sensor Data Streams
Research affiliation: Kiel University > Software Engineering
DOI etc.: 10.1109/BigData47090.2019.9006452
Related URLs:
Projects: TITAN
Date Deposited: 28 Feb 2020 10:40
Last Modified: 28 Feb 2020 10:40

Actions (login required)

View Item View Item

Document Downloads

More statistics for this item...