English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Optimal Transport has sparked vivid interest in recent years, in particularthanks to the Wasserstein distance, which provides a geometrically sensible andintuitive way of comparing probability measures. For computational reasons, theSliced Wasserstein (SW) distance was introduced as an alternative to theWasserstein distance, and has seen uses for training generative Neural Networks(NNs). While convergence of Stochastic Gradient Descent (SGD) has been observedpractically in such a setting, there is to our knowledge no theoreticalguarantee for this observation. Leveraging recent works on convergence of SGDon non-smooth and non-convex functions by Bianchi et al. (2022), we aim tobridge that knowledge gap, and provide a realistic context under whichfixed-step SGD trajectories for the SW loss on NN parameters converge. Moreprecisely, we show that the trajectories approach the set of (sub)-gradientflow equations as the step decreases. Under stricter assumptions, we show amuch stronger convergence result for noised and projected SGD schemes, namelythat the long-run limits of the trajectories approach a set of generalisedcritical points of the loss function.

Convergence of SGD for Training Neural Networks with Sliced Wasserstein Losses