English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recurrent neural networks (RNNs) are powerful models for processingtime-series data, but it remains challenging to understand how they function.Improving this understanding is of substantial interest to both the machinelearning and neuroscience communities. The framework of reverse engineering atrained RNN by linearizing around its fixed points has provided insight, butthe approach has significant challenges. These include difficulty choosingwhich fixed point to expand around when studying RNN dynamics and erroraccumulation when reconstructing the nonlinear dynamics with the linearizeddynamics. We present a new model that overcomes these limitations byco-training an RNN with a novel switching linear dynamical system (SLDS)formulation. A first-order Taylor series expansion of the co-trained RNN and anauxiliary function trained to pick out the RNN's fixed points govern the SLDSdynamics. The results are a trained SLDS variant that closely approximates theRNN, an auxiliary function that can produce a fixed point for each point instate-space, and a trained nonlinear RNN whose dynamics have been regularizedsuch that its first-order terms perform the computation, if possible. Thismodel removes the post-training fixed point optimization and allows us tounambiguously study the learned dynamics of the SLDS at any point instate-space. It also generalizes SLDS models to continuous manifolds ofswitching points while sharing parameters across switches. We validate theutility of the model on two synthetic tasks relevant to previous work reverseengineering RNNs. We then show that our model can be used as a drop-in in morecomplex architectures, such as LFADS, and apply this LFADS hybrid to analyzesingle-trial spiking activity from the motor system of a non-human primate.

Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems