English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Scalability properties of deep neural networks raise key research questions,particularly as the problems considered become larger and more challenging.This paper expands on the idea of conditional computation introduced by Bengio,et. al., where the nodes of a deep network are augmented by a set of gatingunits that determine when a node should be calculated. By factorizing theweight matrix into a low-rank approximation, an estimation of the sign of thepre-nonlinearity activation can be efficiently obtained. For networks usingrectified-linear hidden units, this implies that the computation of a hiddenunit with an estimated negative pre-nonlinearity can be ommitted altogether, asits value will become zero when nonlinearity is applied. For sparse neuralnetworks, this can result in considerable speed gains. Experimental resultsusing the MNIST and SVHN data sets with a fully-connected deep neural networkdemonstrate the performance robustness of the proposed scheme with respect tothe error introduced by the conditional computation process.

Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks