Quality, Reliability, Security and Robustness in Heterogeneous Networks. 7th International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness, QShine 2010, and Dedicated Short Range Communications Workshop, DSRC 2010, Houston, TX, USA, November 17-19, 2010, Revised Selected Papers

Research Article

Efficient Stream Processing in the Cloud

Download
474 downloads
  • @INPROCEEDINGS{10.1007/978-3-642-29222-4_19,
        author={Dung Vu and Vana Kalogeraki and Yannis Drougas},
        title={Efficient Stream Processing in the Cloud},
        proceedings={Quality, Reliability, Security and Robustness in Heterogeneous Networks. 7th International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness, QShine 2010, and Dedicated Short Range Communications Workshop, DSRC 2010, Houston, TX, USA, November 17-19, 2010, Revised Selected Papers},
        proceedings_a={QSHINE},
        year={2012},
        month={10},
        keywords={Stream Processing Peer-to-Peer Distributed Systems},
        doi={10.1007/978-3-642-29222-4_19}
    }
    
  • Dung Vu
    Vana Kalogeraki
    Yannis Drougas
    Year: 2012
    Efficient Stream Processing in the Cloud
    QSHINE
    Springer
    DOI: 10.1007/978-3-642-29222-4_19
Dung Vu1,*, Vana Kalogeraki2,*, Yannis Drougas3,*
  • 1: University of California-Riverside
  • 2: Athens University of Economics and Business
  • 3: Environmental Systems Research Institute
*Contact email: dungv@cs.ucr.edu, vana@aueb.gr, drougas@esri.com

Abstract

In the recent years, many emerging on-line data analysis applications require real-time delivery of the streaming data while dealing with unpredictable increase in the volume of data. In this paper we propose a novel approach for efficient stream processing of bursts in the Cloud. Our approach uses two queues to schedule requests pending execution. When bursts occur, incoming requests that exceed maximum processing capacity of the node, instead of being dropped, are diverted to a secondary queue. Requests in the secondary queue are concurrently scheduled with the primary queue, so that they can be immediately executed whenever the node has any processing power unused as the results of burst fluctuations. With this mechanism, processing power of nodes is fully utilized and the bursts are efficiently accommodated. Our experimental results illustrate the efficiency of our approach.