English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Lateral inhibitory connections have been observed in the cortex of thebiological brain, and has been extensively studied in terms of its role incognitive functions. However, in the vanilla version of backpropagation in deeplearning, all gradients (which can be understood to comprise of both signal andnoise gradients) flow through the network during weight updates. This may leadto overfitting. In this work, inspired by biological lateral inhibition, wepropose Gradient Mask, which effectively filters out noise gradients in theprocess of backpropagation. This allows the learned feature information to bemore intensively stored in the network while filtering out noisy or unimportantfeatures. Furthermore, we demonstrate analytically how lateral inhibition inartificial neural networks improves the quality of propagated gradients. A newcriterion for gradient quality is proposed which can be used as a measureduring training of various convolutional neural networks (CNNs). Finally, weconduct several different experiments to study how Gradient Mask improves theperformance of the network both quantitatively and qualitatively.Quantitatively, accuracy in the original CNN architecture, accuracy afterpruning, and accuracy after adversarial attacks have shown improvements.Qualitatively, the CNN trained using Gradient Mask has developed saliency mapsthat focus primarily on the object of interest, which is useful for dataaugmentation and network interpretability.

Gradient Mask: Lateral Inhibition Mechanism Improves Performance in Artificial Neural Networks