Open AccessA Parallelization Algorithm of Singular Value Decomposition
Author(s) -
Duan Jianfeng,
Xian’e Cun
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1865/4/042004
Subject(s) - singular value decomposition , movielens , computer science , matrix decomposition , algorithm , matrix (chemical analysis) , spark (programming language) , feature (linguistics) , factorization , curse of dimensionality , collaborative filtering , recommender system , artificial intelligence , machine learning , philosophy , quantum mechanics , composite material , programming language , linguistics , eigenvalues and eigenvectors , physics , materials science
Matrix factorization algorithm is one of the recommendable algorithms. In order to tackle the inefficiencies of the traditional matrix factorization algorithm like the slow training time and the insufficient computing resource for the mass data, a parallelization algorithm of singular value decomposition (SVD) under the Spark framework is proposed to perform SVD, standardization, and dimensionality reduction for the user-rating matrix, and obtain the user-feature matrix and project-feature matrix. The recommendation model is obtained by determining the prediction rating. MovieLens data show that this algorithm can significantly shorten the training time of the model, improve the running efficiency of the recommendation algorithms for the mass data, and improve the algorithm accuracy.