Open AccessImproved big data filtering algorithm based on bloom filter
Author(s) -
Yujie Liang,
Ying Yu,
Wenhao Ouyang
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1629/1/012026
Subject(s) - bloom filter , filter (signal processing) , computer science , algorithm , bloom , filter design , adaptive filter , computer vision , optics , physics
This paper focuses on the Bloom Filter deformation algorithm. The storage space occupied by Bloom Filter is independent of elements, which has high space efficiency and low time complexity of insert and query operations. However, as the number of added collections increases, the Bloom Filter’s error rate increases, thus filtering out a lot of non-repeating information. In order to solve this problem, the standard Bloom Filter is improved by introducing an algorithm of the structure of the master-slave Bloom Filter. Only when the master-slave Bloom Filter produces errors, the filter is considered as an error. At the same time, by dynamically increasing the number of filters, the growth of misjudgment rate is delayed. This paper also involves an improved Bloom Filter algorithm to reduce the error rate of the repeated data in the judgment filter.