English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In Score based Generative Modeling (SGMs), the state-of-the-art in generativemodeling, stochastic reverse processes are known to perform better than theirdeterministic counterparts. This paper delves into the heart of thisphenomenon, comparing neural ordinary differential equations (ODEs) and neuralstochastic differential equations (SDEs) as reverse processes. We use a controltheoretic perspective by posing the approximation of the reverse process as atrajectory tracking problem. We analyze the ability of neural SDEs toapproximate trajectories of the Fokker-Planck equation, revealing theadvantages of stochasticity. First, neural SDEs exhibit a powerful regularizingeffect, enabling $L^2$ norm trajectory approximation surpassing the Wassersteinmetric approximation achieved by neural ODEs under similar conditions, evenwhen the reference vector field or score function is not Lipschitz. Applyingthis result, we establish the class of distributions that can be sampled usingscore matching in SGMs, relaxing the Lipschitz requirement on the gradient ofthe data distribution in existing literature. Second, we show that thisapproximation property is preserved when network width is limited to the inputdimension of the network. In this limited width case, the weights act ascontrol inputs, framing our analysis as a controllability problem for neuralSDEs in probability density space. This sheds light on how noise helps to steerthe system towards the desired solution and illuminates the empirical successof stochasticity in generative modeling.

Noise in the reverse process improves the approximation capabilities of diffusion models