English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recurrent neural networks (RNNs), temporal convolutions, and neuraldifferential equations (NDEs) are popular families of deep learning models fortime-series data, each with unique strengths and tradeoffs in modeling powerand computational efficiency. We introduce a simple sequence model inspired bycontrol systems that generalizes these approaches while addressing theirshortcomings. The Linear State-Space Layer (LSSL) maps a sequence $u \mapsto y$by simply simulating a linear continuous-time state-space representation$\dot{x} = Ax + Bu, y = Cx + Du$. Theoretically, we show that LSSL models areclosely related to the three aforementioned families of models and inherittheir strengths. For example, they generalize convolutions to continuous-time,explain common RNN heuristics, and share features of NDEs such as time-scaleadaptation. We then incorporate and generalize recent theory on continuous-timememorization to introduce a trainable subset of structured matrices $A$ thatendow LSSLs with long-range memory. Empirically, stacking LSSL layers into asimple deep neural network obtains state-of-the-art results across time seriesbenchmarks for long dependencies in sequential image classification, real-worldhealthcare regression tasks, and speech. On a difficult speech classificationtask with length-16000 sequences, LSSL outperforms prior approaches by 24accuracy points, and even outperforms baselines that use hand-crafted featureson 100x shorter sequences.

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers