English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A popular track of network compression approach is Quantization awareTraining (QAT), which accelerates the forward pass during the neural networktraining and inference. However, not much prior efforts have been made toquantize and accelerate the backward pass during training, even though thatcontributes around half of the training time. This can be partly attributed tothe fact that errors of low-precision gradients during backward cannot beamortized by the training objective as in the QAT setting. In this work, wepropose to solve this problem by incorporating the gradients into thecomputation graph of the next training iteration via a hypernetwork. Variousexperiments on CIFAR-10 dataset with different CNN network architecturesdemonstrate that our hypernetwork-based approach can effectively reduce thenegative effect of gradient quantization noise and successfully quantizes thegradients to INT4 with only 0.64 accuracy drop for VGG-16 on CIFAR-10.

MetaGrad: Adaptive Gradient Quantization with Hypernetworks