English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper we introduce a theoretical framework for semi-discreteoptimization using ideas from optimal transport. Our primary motivation is inthe field of deep learning, and specifically in the task of neural architecturesearch. With this aim in mind, we discuss the geometric and theoreticalmotivation for new techniques for neural architecture search (in a companionpaper we show that algorithms inspired by our framework are competitive withcontemporaneous methods). We introduce a Riemannian-like metric on the space ofprobability measures over a semi-discrete space $\mathbb{R}^d \times\mathcal{G}$ where $\mathcal{G}$ is a finite weighted graph. With suchRiemmanian structure in hand, we derive formal expressions for the gradientflow of a relative entropy functional, as well as second order dynamics for theoptimization of said energy. Then, with the aim of providing a rigorousmotivation for the gradient flow equations derived formally, we also consideran iterative procedure known as minimizing movement scheme (i.e., ImplicitEuler scheme, or JKO scheme) and apply it to the relative entropy with respectto a suitable cost function. For some specific choices of metric and cost, werigorously show that the minimizing movement scheme of the relative entropyfunctional converges to the gradient flow process provided by the formalRiemannian structure. This flow coincides with a system of reaction-diffusionequations on $\mathbb{R}^d$.

Semi-discrete optimization through semi-discrete optimal transport: a framework for neural architecture search