English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural networks, specifically deep convolutional neural networks, haveachieved unprecedented performance in various computer vision tasks, but therationale for the computations and structures of successful neural networks isnot fully understood. Theories abound for the aptitude of convolutional neuralnetworks for image classification, but less is understood about why such modelswould be capable of complex visual tasks such as inference and anomalyidentification. Here, we propose a sparse coding interpretation of neuralnetworks that have ReLU activation and of convolutional neural networks inparticular. In sparse coding, when the model's basis functions are assumed tobe orthogonal, the optimal coefficients are given by the soft-thresholdfunction of the basis functions projected onto the input image. In anon-negative variant of sparse coding, the soft-threshold function becomes aReLU. Here, we derive these solutions via sparse coding with orthogonal-assumedbasis functions, then we derive the convolutional neural network forwardtransformation from a modified non-negative orthogonal sparse coding model withan exponential prior parameter for each sparse coding coefficient. Next, wederive a complete convolutional neural network without normalization andpooling by adding logistic regression to a hierarchical sparse coding model.Finally we motivate potentially more robust forward transformations bymaintaining sparse priors in convolutional neural networks as well performing astronger nonlinear transformation.

A Sparse Coding Interpretation of Neural Networks and Theoretical Implications