English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

A neural network model of a differential equation, namely neural ODE, hasenabled the learning of continuous-time dynamical systems and probabilisticdistributions with high accuracy. The neural ODE uses the same networkrepeatedly during a numerical integration. The memory consumption of thebackpropagation algorithm is proportional to the number of uses times thenetwork size. This is true even if a checkpointing scheme divides thecomputation graph into sub-graphs. Otherwise, the adjoint method obtains agradient by a numerical integration backward in time. Although this methodconsumes memory only for a single network use, it requires high computationalcost to suppress numerical errors. This study proposes the symplectic adjointmethod, which is an adjoint method solved by a symplectic integrator. Thesymplectic adjoint method obtains the exact gradient (up to rounding error)with memory proportional to the number of uses plus the network size. Theexperimental results demonstrate that the symplectic adjoint method consumesmuch less memory than the naive backpropagation algorithm and checkpointingschemes, performs faster than the adjoint method, and is more robust torounding errors.

Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memory