English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

When factorized approximations are used for variational inference (VI), theytend to underestimate the uncertainty -- as measured in various ways -- of thedistributions they are meant to approximate. We consider two popular ways tomeasure the uncertainty deficit of VI: (i) the degree to which itunderestimates the componentwise variance, and (ii) the degree to which itunderestimates the entropy. To better understand these effects, and therelationship between them, we examine an informative setting where they can beexplicitly (and elegantly) analyzed: the approximation of a Gaussian,~$p$, witha dense covariance matrix, by a Gaussian,~$q$, with a diagonal covariancematrix. We prove that $q$ always underestimates both the componentwise varianceand the entropy of $p$, \textit{though not necessarily to the same degree}.Moreover we demonstrate that the entropy of $q$ is determined by the trade-offof two competing forces: it is decreased by the shrinkage of its componentwisevariances (our first measure of uncertainty) but it is increased by thefactorized approximation which delinks the nodes in the graphical model of $p$.We study various manifestations of this trade-off, notably one where, as thedimension of the problem grows, the per-component entropy gap between $p$ and$q$ becomes vanishingly small even though $q$ underestimates everycomponentwise variance by a constant multiplicative factor. We also use theshrinkage-delinkage trade-off to bound the entropy gap in terms of the problemdimension and the condition number of the correlation matrix of $p$. Finally wepresent empirical results on both Gaussian and non-Gaussian targets, the formerto validate our analysis and the latter to explore its limitations.

The Shrinkage-Delinkage Trade-off: An Analysis of Factorized Gaussian Approximations for Variational Inference