Research Article
Joint Operator Replication and Placement Optimization for Distributed Streaming Applications
@INPROCEEDINGS{10.4108/eai.25-10-2016.2266628, author={Valeria Cardellini and Vincenzo Grassi and Francesco Lo Presti and Matteo Nardelli}, title={Joint Operator Replication and Placement Optimization for Distributed Streaming Applications}, proceedings={10th EAI International Conference on Performance Evaluation Methodologies and Tools}, publisher={ACM}, proceedings_a={VALUETOOLS}, year={2017}, month={5}, keywords={data stream processing operator placement replication qos}, doi={10.4108/eai.25-10-2016.2266628} }- Valeria Cardellini
Vincenzo Grassi
Francesco Lo Presti
Matteo Nardelli
Year: 2017
Joint Operator Replication and Placement Optimization for Distributed Streaming Applications
VALUETOOLS
ACM
DOI: 10.4108/eai.25-10-2016.2266628
Abstract
In the last few years, several processing approaches have emerged to deal with Big Data. Exploiting on-the-fly computation, Data Stream Processing (DSP) applications can process unbounded streams of data to extract valuable information in a near real-time fashion. To keep up with the high volume of daily produced data, the operators that compose a DSP application can be replicated and placed on multiple, possibly distributed, computing nodes, so to process the incoming data flow in parallel.
In this paper, we present Optimal DSP Replication and Placement (ODRP), a unified general formulation of the operator replication and placement problem that takes into account the heterogeneity of application requirements and infrastructural resources. A key feature of ODRP is the joint optimization of the operators replication and their placement. We evaluate the proposed model through a set of numerical experiments that demonstrates its flexibility and the benefits that derive from the joint optimization.
