Premium
A Method for High‐Throughput Deduplication for Primary File Server by Using Prefetch Cache
Author(s) -
KAMEI HITOSHI,
NAKAMURA TAKAKI
Publication year - 2016
Publication title -
electronics and communications in japan
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.131
H-Index - 13
eISSN - 1942-9541
pISSN - 1942-9533
DOI - 10.1002/ecj.11913
Subject(s) - data deduplication , instruction prefetch , computer science , throughput , cache , operating system , database , file server , process (computing) , parallel computing , wireless
SUMMARY We propose a method of high‐throughput file‐level deduplication for primary file servers, called partial data background prefetch (PDBP). To achieve high throughput of deduplication, the method reduces the number of disk I/Os issued during deduplication process. Before running deduplication process, the proposed method prefetches a part of data of shred files referred by deduplicated files. After that, the method processes the files that are larger than a file‐size threshold defined by administrators. In this paper, we evaluate a deduplication processing time by using a simulation model of PDBP. Consequently, we confirm that the processing time of PDBP is reduced by about 50% compared to a conventional file deduplication method when the threshold is set to 4 KB.