Premium
Optimizing MapReduce for energy efficiency
Author(s) -
Tiwari Nidhi,
Bellur Umesh,
Sarkar Santonu,
Indrawan Maria
Publication year - 2018
Publication title -
software: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.437
H-Index - 70
eISSN - 1097-024X
pISSN - 0038-0644
DOI - 10.1002/spe.2599
Subject(s) - configurator , computer science , efficient energy use , energy consumption , cluster (spacecraft) , process (computing) , fault tolerance , distributed computing , energy (signal processing) , big data , operating system , engineering , statistics , mathematics , marketing , electrical engineering , business
Summary The efficient use of energy is essential to address concerns of cost and sustainability. Many data centers contain MapReduce clusters to process Big Data applications. A large number of machines and fault tolerance capabilities make MapReduce clusters energy inefficient. In this paper, we present a Configurator based on performance and energy models to improve the energy efficiency of MapReduce systems. Our solution is novel as it takes into account the dependence of the performance and energy consumption of a cluster on MapReduce parameters. While this dependence is known, we are the first to model it and design a Configurator to optimize these parameter settings for maximizing the energy efficiency of MapReduce systems. Our empirical evaluations show that the Configurator can result in up to 50% improvement in the energy efficiency of typical MapReduce applications in two architecturally different clusters.