Premium
Acceleration of the EM Algorithm by using Quasi‐Newton Methods
Author(s) -
Jamshidian Mortaza,
Jennrich Robert I.
Publication year - 1997
Publication title -
journal of the royal statistical society: series b (statistical methodology)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.523
H-Index - 137
eISSN - 1467-9868
pISSN - 1369-7412
DOI - 10.1111/1467-9868.00083
Subject(s) - acceleration , algorithm , convergence (economics) , computer science , covariance , inverse , simplicity , newton's method , inverse problem , mathematics , mathematical optimization , statistics , mathematical analysis , physics , geometry , classical mechanics , nonlinear system , quantum mechanics , economics , economic growth , philosophy , epistemology
The EM algorithm is a popular method for maximum likelihood estimation. Its simplicity in many applications and desirable convergence properties make it very attractive. Its sometimes slow convergence, however, has prompted researchers to propose methods to accelerate it. We review these methods, classifying them into three groups: pure , hybrid and EM‐type accelerators. We propose a new pure and a new hybrid accelerator both based on quasi‐Newton methods and numerically compare these and two other quasi‐Newton accelerators. For this we use examples in each of three areas: Poisson mixtures, the estimation of covariance from incomplete data and multivariate normal mixtures. In these comparisons, the new hybrid accelerator was fastest on most of the examples and often dramatically so. In some cases it accelerated the EM algorithm by factors of over 100. The new pure accelerator is very simple to implement and competed well with the other accelerators. It accelerated the EM algorithm in some cases by factors of over 50. To obtain standard errors, we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator. A numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes.