Premium
A Unifying Framework and Comparison of Algorithms for Non‐negative Matrix Factorisation
Author(s) -
Hobolth Asger,
Guo Qianyun,
Kousholt Astrid,
Jensen Jens Ledet
Publication year - 2020
Publication title -
international statistical review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.051
H-Index - 54
eISSN - 1751-5823
pISSN - 0306-7734
DOI - 10.1111/insr.12331
Subject(s) - non negative matrix factorization , coordinate descent , algorithm , gradient descent , projection (relational algebra) , mathematics , iterative method , cone (formal languages) , computer science , matrix decomposition , artificial intelligence , artificial neural network , eigenvalues and eigenvectors , physics , quantum mechanics
Summary Non‐negative matrix factorisation (NMF) is an increasingly popular unsupervised learning method. However, parameter estimation in the NMF model is a difficult high‐dimensional optimisation problem. We consider algorithms of the alternating least squares type. Solutions to the least squares problem fall in two categories. The first category is iterative algorithms, which include algorithms such as the majorise–minimise (MM) algorithm, coordinate descent, gradient descent and the Févotte‐Cemgil expectation–maximisation (FC‐EM) algorithm. We introduce a new family of iterative updates based on a generalisation of the FC‐EM algorithm. The coordinate descent, gradient descent and FC‐EM algorithms are special cases of this new EM family of iterative procedures. Curiously, we show that the MM algorithm is never a member of our general EM algorithm. The second category is based on cone projection. We describe and prove a cone projection algorithm tailored to the non‐negative least square problem. We compare the algorithms on a test case and on the problem of identifying mutational signatures in human cancer. We generally find that cone projection is an attractive choice. Furthermore, in the cancer application, we find that a mix‐and‐match strategy performs better than running each algorithm in isolation.