Research Article
A Topology-Aware Scheduling Strategy for Distributed Stream Computing System
@INPROCEEDINGS{10.1007/978-3-030-93479-8_8, author={Bo Li and Dawei Sun and Vinh Loi Chau and Rajkumar Buyya}, title={A Topology-Aware Scheduling Strategy for Distributed Stream Computing System}, proceedings={Broadband Communications, Networks, and Systems. 12th EAI International Conference, BROADNETS 2021, Virtual Event, October 28--29, 2021, Proceedings}, proceedings_a={BROADNETS}, year={2022}, month={1}, keywords={Stream computing Big data system Topology-aware Scheduling Graph division}, doi={10.1007/978-3-030-93479-8_8} }- Bo Li
Dawei Sun
Vinh Loi Chau
Rajkumar Buyya
Year: 2022
A Topology-Aware Scheduling Strategy for Distributed Stream Computing System
BROADNETS
Springer
DOI: 10.1007/978-3-030-93479-8_8
Abstract
Reducing latency has become the focus of task scheduling research in distributed big data stream computing systems. Currently, most task schedulers in big data stream computing systems mainly focus on tasks assignment and implicitly ignore task topology which can have significant impact on the latency and energy efficiency. This paper proposes a topology-aware scheduling strategy to reduce the processing latency of stream processing systems. We construct the data stream graph as a directed acyclic graph and then, divide it using the graph Laplace algorithm. On the divided graph, tasks will be assigned with a low-latency scheduling strategy. We also provide a computing node selection strategy, which enables the system to run tasks on the topology with the least number of computing nodes. Based on this scheduling strategy, the tasks of the data stream graph can be redistributed and the scheduling mechanism can be optimized to minimize the system latency. The experimental results demonstrate the efficiency and effectiveness of the proposed strategy.
