Research Article
A Data Stream Prediction Strategy for Elastic Stream Computing Systems
@INPROCEEDINGS{10.1007/978-3-030-93479-8_9, author={Hanchu Zhang and Dawei Sun and Atul Sajjanhar and Rajkumar Buyya}, title={A Data Stream Prediction Strategy for Elastic Stream Computing Systems}, proceedings={Broadband Communications, Networks, and Systems. 12th EAI International Conference, BROADNETS 2021, Virtual Event, October 28--29, 2021, Proceedings}, proceedings_a={BROADNETS}, year={2022}, month={1}, keywords={Data stream prediction Resource scheduling Stream computing Back propagation neural network Storm}, doi={10.1007/978-3-030-93479-8_9} }- Hanchu Zhang
Dawei Sun
Atul Sajjanhar
Rajkumar Buyya
Year: 2022
A Data Stream Prediction Strategy for Elastic Stream Computing Systems
BROADNETS
Springer
DOI: 10.1007/978-3-030-93479-8_9
Abstract
In a distributed stream processing system, elastic resource provisioning/scheduling is the main factor that affects system performance and limits system applications. However, in the data stream computing platform, resource allocation is often suboptimal due to the large fluctuations of the data stream rate, which creates a performance bottleneck for the cluster. In this paper, we propose a data stream prediction strategy (Dp-Stream) for elastic computing system to mitigate the resource allocation issue. First, we establish a back propagation (BP) neural network prediction model based on genetic simulated annealing algorithm to predict the trend of the data stream rate in the next time window of the cluster; second, according to the time latency, the estimation model adjusts the resources allocated to the critical operations of the critical path in the Directed Acyclic Graph (DAG) and finally, the resource communication cost is optimized. We evaluate the prediction accuracy and system latency of the proposed scheduling strategy in Storm. The experimental results prove the feasibility and effectiveness of the proposed strategy.
