Open AccessImproving Mapreduce Process By Introducing Aggregator Repartition Data for Big Data Analytics
Author(s) -
K. Sathesh Kumar,
S. Ramkumar,
K. Shankar,
M. Ilayaraja
Publication year - 2019
Publication title -
international journal of engineering and advanced technology
Language(s) - English
Resource type - Journals
ISSN - 2249-8958
DOI - 10.35940/ijeat.a1109.1291s419
Subject(s) - reducer , news aggregator , computer science , computation , big data , key (lock) , data mining , process (computing) , algorithm , operating system , engineering , civil engineering
This work suggested data aggregator is used in between the mapper and reducer to enhance the performance of MapReduce. Initially the massive amount of data is partitioned into number of subset of data through the n number of independent mappers and it produces key value pairs for each partitioned data. Then the key value pairs are fed into aggregator where the data from different mappers are combining with smaller amount than the input. Followed by data aggregation data de duplication is carried over then repartition the data based on content, computation and network aware of data. Finally reducer merges the data to produce the final output, the proposed Content, computation and Network Aware (CCNA) MapReducer is compared with the existing Content Aware (CA) MapReducer and Content, computation Aware (CCA) MapReducer.