English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper introduces novel results for the score function gradient estimatorof the importance weighted variational bound (IWAE). We prove that in the limitof large $K$ (number of importance samples) one can choose the control variatesuch that the Signal-to-Noise ratio (SNR) of the estimator grows as $\sqrt{K}$.This is in contrast to the standard pathwise gradient estimator where the SNRdecreases as $1/\sqrt{K}$. Based on our theoretical findings we develop a novelcontrol variate that extends on VIMCO. Empirically, for the training of bothcontinuous and discrete generative models, the proposed method yields superiorvariance reduction, resulting in an SNR for IWAE that increases with $K$without relying on the reparameterization trick. The novel estimator iscompetitive with state-of-the-art reparameterization-free gradient estimatorssuch as Reweighted Wake-Sleep (RWS) and the thermodynamic variational objective(TVO) when training generative models.

Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Bounds