Premium
A MapReduce‐supported network structure for data centers
Author(s) -
Ding Zeliu,
Guo Deke,
Liu Xue,
Luo Xueshan,
Chen Guihai
Publication year - 2011
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.1791
Subject(s) - computer science , data structure , tree (set theory) , network structure , tree structure , big data , data center , distributed computing , data mining , computer network , operating system , mathematical analysis , mathematics
SUMMARY Several novel data center network structures have been proposed to improve the topological properties of data centers. A common characteristic of these structures is that they are designed for supporting general applications and services. Consequently, these structures do not match well with the specific requirements of some dedicated applications. In this paper, we propose a hyper‐fat‐tree network (HFN): a novel data center structure for MapReduce, a well‐known distributed data processing application. HFN possesses the advanced characteristics of BCube as well as fat‐tree structures and naturally supports MapReduce. We then address several challenging issues that face HFN in supporting MapReduce. Mathematical analysis and comprehensive evaluation show that HFN possesses excellent properties and is indeed a viable structure for MapReduce in practice. Copyright © 2011 John Wiley & Sons, Ltd.