English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Variational autoencoders (VAEs) face a notorious problem wherein thevariational posterior often aligns closely with the prior, a phenomenon knownas posterior collapse, which hinders the quality of representation learning. Tomitigate this problem, an adjustable hyperparameter $\beta$ and a strategy forannealing this parameter, called KL annealing, are proposed. This studypresents a theoretical analysis of the learning dynamics in a minimal VAE. Itis rigorously proved that the dynamics converge to a deterministic processwithin the limit of large input dimensions, thereby enabling a detaileddynamical analysis of the generalization error. Furthermore, the analysis showsthat the VAE initially learns entangled representations and gradually acquiresdisentangled representations. A fixed-point analysis of the deterministicprocess reveals that when $\beta$ exceeds a certain threshold, posteriorcollapse becomes inevitable regardless of the learning period. Additionally,the superfluous latent variables for the data-generative factors lead tooverfitting of the background noise; this adversely affects both generalizationand learning convergence. The analysis further unveiled that appropriatelytuned KL annealing can accelerate convergence.

Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing