Open AccessOptimal Design Method of Merging Disk Files Based on Hot Data
Author(s) -
Qian Lin,
Mingjie Xu,
Jun Yu,
Weichao Li,
Hao Yuan,
Jingbin Ren,
Zhu Mei,
Hengmao Pang
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1549/2/022075
Subject(s) - computer science , hotspot (geology) , subsequence , big data , database , computer data storage , data mining , computer hardware , mathematical analysis , mathematics , geophysics , bounded function , geology
In recent years, with the rapid development of big data technology, users are more and more inclined to solve the problems of large amount of data and complex business scenarios with big data platform. When the system is faced with the scene of data hotspot and frequent data modification, it will produce a lot of useless data and cause data hotspot problems, which make a lot of access to the level of disk storage. Unreasonable data allocation of HFile will affect I/O performance and reduce system availability. This thesis proposes a comparison strategy based hot data. According to the frequency of data access, this stategy change the selection method of merging subsequence of Exploring Compaction Policy, so as to achieve a more suitable effect for hot data query business.