English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Optimization with noisy gradients has become ubiquitous in statistics andmachine learning. Reparameterization gradients, or gradient estimates computedvia the "reparameterization trick," represent a class of noisy gradients oftenused in Monte Carlo variational inference (MCVI). However, when these gradientestimators are too noisy, the optimization procedure can be slow or fail toconverge. One way to reduce noise is to use more samples for the gradientestimate, but this can be computationally expensive. Instead, we view the noisygradient as a random variable, and form an inexpensive approximation of thegenerating procedure for the gradient sample. This approximation has highcorrelation with the noisy gradient by construction, making it a useful controlvariate for variance reduction. We demonstrate our approach on non-conjugatemulti-level hierarchical models and a Bayesian neural net where we observedgradient variance reductions of multiple orders of magnitude (20-2,000x).

Reducing Reparameterization Gradient Variance