English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we present some applications of random matrix theory for thetraining of deep neural networks. Recently, random matrix theory (RMT) has beenapplied to the overfitting problem in deep learning. Specifically, it has beenshown that the spectrum of the weight layers of a deep neural network (DNN) canbe studied and understood using techniques from RMT. In this work, these RMTtechniques will be used to determine which and how many singular values shouldbe removed from the weight layers of a DNN during training, via singular valuedecomposition (SVD), so as to reduce overfitting and increase accuracy. We showthe results on a simple DNN model trained on MNIST. In general, thesetechniques may be applied to any fully connected layer of a pretrained DNN toreduce the number of parameters in the layer while preserving and sometimesincreasing the accuracy of the DNN.

Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting