English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Stein Variational Gradient Descent (SVGD) is a popular variational inferencealgorithm which simulates an interacting particle system to approximatelysample from a target distribution, with impressive empirical performance acrossvarious domains. Theoretically, its population (i.e, infinite-particle) limitdynamics is well studied but the behavior of SVGD in the finite-particle regimeis much less understood. In this work, we design two computationally efficientvariants of SVGD, namely VP-SVGD and GB-SVGD, with provably fastfinite-particle convergence rates. We introduce the notion of virtual particlesand develop novel stochastic approximations of population-limit SVGD dynamicsin the space of probability measures, which are exactly implementable using afinite number of particles. Our algorithms can be viewed as specificrandom-batch approximations of SVGD, which are computationally more efficientthan ordinary SVGD. We show that the $n$ particles output by VP-SVGD andGB-SVGD, run for $T$ steps with batch-size $K$, are at-least as good as i.i.dsamples from a distribution whose Kernel Stein Discrepancy to the target is atmost $O\left(\tfrac{d^{1/3}}{(KT)^{1/6}}\right)$ under standard assumptions.Our results also hold under a mild growth condition on the potential function,which is much weaker than the isoperimetric (e.g. Poincare Inequality) orinformation-transport conditions (e.g. Talagrand's Inequality $\mathsf{T}_1$)generally considered in prior works. As a corollary, we consider theconvergence of the empirical measure (of the particles output by VP-SVGD andGB-SVGD) to the target distribution and demonstrate a double exponentialimprovement over the best known finite-particle analysis of SVGD. Beyond this,our results present the first known oracle complexities for this setting withpolynomial dimension dependence.

Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation