Open AccessFast Frequent Item Mining from Big Data using Map Reduce and Bit Vectors
Author(s) -
S Thirumaran.
Publication year - 2021
Publication title -
türk bilgisayar ve matematik eğitimi dergisi
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.218
H-Index - 3
ISSN - 1309-4653
DOI - 10.17762/turcomat.v12i2.1525
Subject(s) - computer science , data mining , dual (grammatical number) , hash function , big data , apriori algorithm , representation (politics) , algorithm , association rule learning , art , literature , computer security , politics , political science , law
One of the most important areas that are constantly being focused recently is the big data and mining frequent patterns from them is an interesting vertical which is perpetually being evolved and gained plethora of attention among the research fraternities. Generally, the data is mined with the aid of Apriori based algorithms, tree based algorithm and hash based algorithm but most of these existing algorithms suffer many snags and limitations. This paper proposes a new method that overrides and overcomes the most common problems related to speed, memory consumption and search space. The algorithm named Dual Mine employs binary vector representation and vertical data representations in the map reduce and then discover the most patterns from the large data sets. The Dual mine algorithm is then compared with some of the existing algorithms to determine the efficiency of the proposed algorithm and from the experimental results it is quite evident that the proposed algorithm “Dual Mine” outscored the other algorithms by a big magnitude with respect to speed and memory.