English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advances in the sparse neural network literature have made it possibleto prune many large feed forward and convolutional networks with only a smallquantity of data. Yet, these same techniques often falter when applied to theproblem of recovering sparse recurrent networks. These failures arequantitative: when pruned with recent techniques, RNNs typically obtain worseperformance than they do under a simple random pruning scheme. The failures arealso qualitative: the distribution of active weights in a pruned LSTM or GRUnetwork tend to be concentrated in specific neurons and gates, and not welldispersed across the entire architecture. We seek to rectify both thequantitative and qualitative issues with recurrent network pruning byintroducing a new recurrent pruning objective derived from the spectrum of therecurrent Jacobian. Our objective is data efficient (requiring only 64 datapoints to prune the network), easy to implement, and produces 95% sparse GRUsthat significantly improve on existing baselines. We evaluate on sequentialMNIST, Billion Words, and Wikitext.

One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation