English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We compare the discretize-optimize (Disc-Opt) and optimize-discretize(Opt-Disc) approaches for time-series regression and continuous normalizingflows (CNFs) using neural ODEs. Neural ODEs are ordinary differential equations(ODEs) with neural network components. Training a neural ODE is an optimalcontrol problem where the weights are the controls and the hidden features arethe states. Every training iteration involves solving an ODE forward andanother backward in time, which can require large amounts of computation, time,and memory. Comparing the Opt-Disc and Disc-Opt approaches in imageclassification tasks, Gholami et al. (2019) suggest that Disc-Opt is preferabledue to the guaranteed accuracy of gradients. In this paper, we extend thecomparison to neural ODEs for time-series regression and CNFs. Unlike inclassification, meaningful models in these tasks must also satisfy additionalrequirements beyond accurate final-time output, e.g., the invertibility of theCNF. Through our numerical experiments, we demonstrate that with carefulnumerical treatment, Disc-Opt methods can achieve similar performance asOpt-Disc at inference with drastically reduced training costs. Disc-Opt reducedcosts in six out of seven separate problems with training time reductionranging from 39% to 97%, and in one case, Disc-Opt reduced training from ninedays to less than one day.

Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows