English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we study the problem of improving computational resourceutilization of neural networks. Deep neural networks are usuallyover-parameterized for their tasks in order to achieve good performances, thusare likely to have underutilized computational resources. This observationmotivates a lot of research topics, e.g. network pruning, architecture search,etc. As models with higher computational costs (e.g. more parameters or morecomputations) usually have better performances, we study the problem ofimproving the resource utilization of neural networks so that their potentialscan be further realized. To this end, we propose a novel optimization methodnamed Neural Rejuvenation. As its name suggests, our method detects deadneurons and computes resource utilization in real time, rejuvenates deadneurons by resource reallocation and reinitialization, and trains them with newtraining schemes. By simply replacing standard optimizers with NeuralRejuvenation, we are able to improve the performances of neural networks by avery large margin while using similar training efforts and maintaining theiroriginal resource usages.

Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization