Open AccessImprovement and parallelization of k-means clustering algorithm based on Spark platform
Author(s) -
Zhihua Xiang
Publication year - 2019
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1345/5/052075
Subject(s) - cluster analysis , spark (programming language) , computer science , similarity (geometry) , data mining , algorithm , machine learning , artificial intelligence , image (mathematics) , programming language
Clustering algorithm analysis is a very common data mining technology in the current industry research, and has been widely promoted and applied in many social marketing and other related industries. It can not only classify, identify and optimize customers accurately, but also design documents by various data processing methods. K-means clustering algorithm is based on the relevant ideological content of “birds of a feather flock together” to split and synthesize data, so that the similarity of data between different clusters can reach the highest. This paper discusses and improves the algorithm based on Spark.