English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

To understand the fundamental trade-offs between training stability, temporaldynamics and architectural complexity of recurrent neural networks~(RNNs), wedirectly analyze RNN architectures using numerical methods of ordinarydifferential equations~(ODEs). We define a general family of RNNs--theODERNNs--by relating the composition rules of RNNs to integration methods ofODEs at discrete time steps. We show that the degree of RNN's functionalnonlinearity $n$ and the range of its temporal memory $t$ can be mapped to thecorresponding stage of Runge-Kutta recursion and the order of time-derivativeof the ODEs. We prove that popular RNN architectures, such as LSTM and URNN,fit into different orders of $n$-$t$-ODERNNs. This exact correspondence betweenRNN and ODE helps us to establish the sufficient conditions for RNN trainingstability and facilitates more flexible top-down designs of new RNNarchitectures using large varieties of toolboxes from numerical integration ofODEs. We provide such an example: Quantum-inspired Universal computing NeuralNetwork~(QUNN), which reduces the required number of training parameters frompolynomial in both data length and temporal memory length to only linear intemporal memory length.

Recurrent Neural Networks in the Eye of Differential Equations