10th EAI International Conference on Performance Evaluation Methodologies and Tools

Research Article

Joint Operator Replication and Placement Optimization for Distributed Streaming Applications

  • @INPROCEEDINGS{10.4108/eai.25-10-2016.2266628,
        author={Valeria Cardellini and Vincenzo Grassi and Francesco Lo Presti and Matteo Nardelli},
        title={Joint Operator Replication and Placement Optimization for Distributed Streaming Applications},
        proceedings={10th EAI International Conference on Performance Evaluation Methodologies and Tools},
        publisher={ACM},
        proceedings_a={VALUETOOLS},
        year={2017},
        month={5},
        keywords={data stream processing operator placement replication qos},
        doi={10.4108/eai.25-10-2016.2266628}
    }
    
  • Valeria Cardellini
    Vincenzo Grassi
    Francesco Lo Presti
    Matteo Nardelli
    Year: 2017
    Joint Operator Replication and Placement Optimization for Distributed Streaming Applications
    VALUETOOLS
    ACM
    DOI: 10.4108/eai.25-10-2016.2266628
Valeria Cardellini1, Vincenzo Grassi1, Francesco Lo Presti1, Matteo Nardelli1,*
  • 1: University of Rome Tor Vergata, Italy
*Contact email: nardelli@ing.uniroma2.it

Abstract

In the last few years, several processing approaches have emerged to deal with Big Data. Exploiting on-the-fly computation, Data Stream Processing (DSP) applications can process unbounded streams of data to extract valuable information in a near real-time fashion. To keep up with the high volume of daily produced data, the operators that compose a DSP application can be replicated and placed on multiple, possibly distributed, computing nodes, so to process the incoming data flow in parallel.

In this paper, we present Optimal DSP Replication and Placement (ODRP), a unified general formulation of the operator replication and placement problem that takes into account the heterogeneity of application requirements and infrastructural resources. A key feature of ODRP is the joint optimization of the operators replication and their placement. We evaluate the proposed model through a set of numerical experiments that demonstrates its flexibility and the benefits that derive from the joint optimization.