Open AccessA DEEP DIVE INTO LOAD BALANCING TOOLS FOR HADOOP APPLICATION MANAGEMENT
Author(s) -
Kaveri T Hombal,
Anjan K Koundinya
Publication year - 2019
Publication title -
international journal of engineering applied science and technology
Language(s) - English
Resource type - Journals
ISSN - 2455-2143
DOI - 10.33564/ijeast.2019.v04i08.018
Subject(s) - computer science , load balancing (electrical power) , operating system , geology , geodesy , grid
Hadoop has become an important tool for the researchers and scientists in order to store and analyze huge amount of data. This huge data is placed in Hadoop with the help of Hadoop Distributed File System (HDFS). Block placement policy is employed in HDFS to split a really huge file into blocks and place these block across the cluster in an exceedingly distributed manner. Basically, Hadoop and HDFS are designed to works expeditiously on the consistent cluster. However during this era of networking, we cannot think about having solely a cluster of consistent nodes. So, there's the necessity of storage policy which will work expeditiously on each consistent still because of the heterogeneous cluster. Thus, the need of applications which will be executing in a time-efficiently manner and supporting consistent still because the heterogeneous setting will be sufficient. In Hadoop data1. MapReduce is programming framework for writing MapReduce applications which enables them to run on the distributed platform in parallel. MapReduce permits the applications to run on Hadoop environment. Hadoop uses HDFS block placement policy to place the data blocks on nodes. Hadoop cluster gets unbalanced every now and then, because of overutilization of few nodes against the less used nodes or recently created other new nodes with no blocks hold on them. To resolve this case, Hadoop encompasses an inherent tool known as HDFS Balancer. KeyWordsHadoop, Load Balancing, Data Blocks, HDFS, Storage.