English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Bayesian inference allows to obtain useful information on the parameters ofmodels, either in computational statistics or more recently in the context ofBayesian Neural Networks. The computational cost of usual Monte Carlo methodsfor sampling posterior laws in Bayesian inference scales linearly with thenumber of data points. One option to reduce it to a fraction of this cost is toresort to mini-batching in conjunction with unadjusted discretizations ofLangevin dynamics, in which case only a random fraction of the data is used toestimate the gradient. However, this leads to an additional noise in thedynamics and hence a bias on the invariant measure which is sampled by theMarkov chain. We advocate using the so-called Adaptive Langevin dynamics, whichis a modification of standard inertial Langevin dynamics with a dynamicalfriction which automatically corrects for the increased noise arising frommini-batching. We investigate the practical relevance of the assumptionsunderpinning Adaptive Langevin (constant covariance for the estimation of thegradient, Gaussian minibatching noise), which are not satisfied in typicalmodels of Bayesian inference, and quantify the bias induced by minibatching inthis case. We also suggest a possible extension of AdL to further reduce thebias on the posterior distribution, by considering a dynamical frictiondepending on the current value of the parameter to sample.

Quantifying the mini-batching error in Bayesian inference for Adaptive Langevin dynamics