Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets | Zendy

Hrishikesh Gadre | Zendy; Iván Rodero | Zendy; Manish Parashar | Zendy

AI Assistant Blog Pricing

Home ZAIA Blog

Open Access

Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets

Author(s) -

Hrishikesh Gadre,

Iván Rodero,

Manish Parashar

Publication year - 2011

Publication title -

acm sigmetrics performance evaluation review

Language(s) - English

Resource type - Journals

SCImago Journal Rank - 0.223

H-Index - 80

eISSN - 1557-9484

pISSN - 0163-5999

DOI - 10.1145/2160803.2160876

Subject(s) - computer science , scheduling (production processes) , map reduce , distributed computing , programming paradigm , task (project management) , cluster (spacecraft) , big data , parallel computing , data mining , operating system , mathematical optimization , mathematics , management , economics , programming language

In this paper, we investigate real-world scenarios in which MapReduce programming model and specifically Hadoop framework could be used for processing large-scale, geographically scattered datasets. We propose an Adaptive Reduce Task Scheduling (ARTS) algorithm and evaluate it on a distributed Hadoop cluster involving multiple datacenters as well as the on a shared Hadoop cluster. The evaluation demonstrates that the ARTS algorithm outperforms the default Reduce phase scheduling algorithm in Hadoop framework.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.

Having issues? You can contact us here

Accelerating Research