Premium
In‐network block repairing for erasure coding storage systems
Author(s) -
Xia Junxu,
Guo Deke,
Cheng Geyao
Publication year - 2019
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.5432
Subject(s) - erasure code , computer science , distributed data store , linear network coding , distributed computing , computer data storage , computer network , overhead (engineering) , erasure , coding (social sciences) , node (physics) , block (permutation group theory) , linked list , decoding methods , algorithm , engineering , computer hardware , network packet , operating system , statistics , geometry , mathematics , programming language , structural engineering
Summary In the erasure coding storage system, it is necessary to extract multiple data blocks from other remaining storage nodes to a new node when a storage node fails, which repairs the failed data block satisfactorily. However, this would incur the incast problem at this new node. The existing solutions for the repair process in the incast problem mainly rely on path planning and resource allocation. Although these solutions improve the performance of repairing the failed data blocks, they still waste a large amount of storage and bandwidth resources unavoidably. In this paper, we propose the incast problem to be resolved economically via the in‐network aggregation. Specifically, we assume that the switches in data centers have certain data processing capabilities and can aggregate data flows efficiently. Thereafter, we propose a set of in‐network methods to repair a failed data block in the erasure coding storage systems, taking the fat‐tree data center as an example. Thus, the incast problem can be solved effectively during the data transmission process. Compared with the prior methods, our approach effectively avoids the overhead of extra path computing, as well as significantly reduces the link cost of repairing data blocks, while promising similar or faster repair speed.