English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Expectation-Maximization (EM) is a prominent approach for parameterestimation of hidden (aka latent) variable models. Given the full batch ofdata, EM forms an upper-bound of the negative log-likelihood of the model ateach iteration and updates to the minimizer of this upper-bound. We firstprovide a "model level" interpretation of the EM upper-bound as sum of relativeentropy divergences to a set of singleton models, induced by the set ofobservations. Our alternative motivation unifies the "observation level" andthe "model level" view of the EM. As a result, we formulate an online versionof the EM algorithm by adding an analogous inertia term which corresponds tothe relative entropy divergence to the old model. Our motivation is more widelyapplicable than the previous approaches and leads to simple online updates formixture of exponential distributions, hidden Markov models, and the first knownonline update for Kalman filters. Additionally, the finite sample form of theinertia term lets us derive online updates when there is no closed-formsolution. Finally, we extend the analysis to the distributed setting where wemotivate a systematic way of combining multiple hidden variable models.Experimentally, we validate the results on synthetic as well as real-worlddatasets.

Divergence-Based Motivation for Online EM and Combining Hidden Variable Models