Premium
Scalable RDF data compression with MapReduce
Author(s) -
Urbani Jacopo,
Maassen Jason,
Drost Niels,
Seinstra Frank,
Bal Henri
Publication year - 2013
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.2840
Subject(s) - computer science , rdf , scalability , data compression , encoding (memory) , set (abstract data type) , compression ratio , compression (physics) , data mining , data set , database , semantic web , distributed computing , information retrieval , algorithm , artificial intelligence , programming language , engineering , composite material , materials science , automotive engineering , internal combustion engine
SUMMARY The Semantic Web contains many billions of statements, which are released using the resource description framework (RDF) data model. To better handle these large amounts of data, high performance RDF applications must apply a compression technique. Unfortunately, because of the large input size, even this compression is challenging. In this paper, we propose a set of distributed MapReduce algorithms to efficiently compress and decompress a large amount of RDF data. Our approach uses a dictionary encoding technique that maintains the structure of the data. We highlight the problems of distributed data compression and describe the solutions that we propose. We have implemented a prototype using the Hadoop framework, and evaluate its performance. We show that our approach is able to efficiently compress a large amount of data and scales linearly on both input size and number of nodes. Copyright © 2012 John Wiley & Sons, Ltd.