ew 18(20): e15

Research Article

DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS

Download77 downloads
  • @ARTICLE{10.4108/eai.12-9-2018.155745,
        author={Dr. Kiran Kumar Pulamolu and  Dr. D. Venkata  Subramanian and Dr Krishnaraj},
        title={DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS},
        journal={EAI Endorsed Transactions on Energy Web and Information Technologies},
        volume={5},
        number={20},
        publisher={EAI},
        journal_a={EW},
        year={2018},
        month={9},
        keywords={Distributed Cluster, Resource Fairness, Resource Sharing, Hierarchical Cluster, MapReduce},
        doi={10.4108/eai.12-9-2018.155745}
    }
    
  • Dr. Kiran Kumar Pulamolu
    Dr. D. Venkata Subramanian
    Dr Krishnaraj
    Year: 2018
    DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS
    EW
    EAI
    DOI: 10.4108/eai.12-9-2018.155745
Dr. Kiran Kumar Pulamolu1,*, Dr. D. Venkata Subramanian2, Dr Krishnaraj1
  • 1: Professor, Sasi Institute of Technology and Engineering
  • 2: Professor, School of Computer Science, Hindustan Institute of Technology & Science, Chennai
*Contact email: kiran@sasi.ac.in

Abstract

MapReduce is a popular, open source programming paradigm to handle big data which is an industry standard large scale data processing system used by many companies like Yahoo, Google, Facebook, etc. The YARN framework uses low resource fairness algorithms such as FIFO, Capacity, Fair, DRF scheduler, whereas these schedulers are not suitable for heterogeneous Hadoop clusters. Therefore, an Enhanced Combined Regression Ranking (eCRRYARN) algorithm was proposed to enhance resource fairness. The proposed algorithm uses linear regression model to estimate the expected resources to be availed by the tenants. The order ranking is given to the estimated resource and the resources shared as per the ranking provided. Hence, the Hierarchical Hadoop Cluster Resource Sharing (HHCRS) algorithm has been adopted for hierarchical distributed cluster aiming to design a cost effective cluster for organization which is spread across the globe.