English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Score-based diffusion models have emerged as one of the most promisingframeworks for deep generative modelling, due to their state-of-the artperformance in many generation tasks while relying on mathematical foundationssuch as stochastic differential equations (SDEs) and ordinary differentialequations (ODEs). Empirically, it has been reported that ODE based samples areinferior to SDE based samples. In this paper we rigorously describe the rangeof dynamics and approximations that arise when training score-based diffusionmodels, including the true SDE dynamics, the neural approximations, the variousapproximate particle dynamics that result, as well as their associatedFokker--Planck equations and the neural network approximations of theseFokker--Planck equations. We systematically analyse the difference between theODE and SDE dynamics of score-based diffusion models, and link it to anassociated Fokker--Planck equation. We derive a theoretical upper bound on theWasserstein 2-distance between the ODE- and SDE-induced distributions in termsof a Fokker--Planck residual. We also show numerically that conventionalscore-based diffusion models can exhibit significant differences between ODE-and SDE-induced distributions which we demonstrate using explicit comparisons.Moreover, we show numerically that reducing the Fokker--Planck residual byadding it as an additional regularisation term leads to closing the gap betweenODE- and SDE-induced distributions. Our experiments suggest that thisregularisation can improve the distribution generated by the ODE, however thatthis can come at the cost of degraded SDE sample quality.

Closing the ODE-SDE gap in score-based diffusion models through the Fokker-Planck equation