English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Quantization techniques applied to the inference of deep neural networks haveenabled fast and efficient execution on resource-constraint devices. Thesuccess of quantization during inference has motivated the academic communityto explore fully quantized training, i.e. quantizing back-propagation as well.However, effective gradient quantization is still an open problem. Gradientsare unbounded and their distribution changes significantly during training,which leads to the need for dynamic quantization. As we show, dynamicquantization can lead to significant memory overhead and additional datatraffic slowing down training. We propose a simple alternative to dynamicquantization, in-hindsight range estimation, that uses the quantization rangesestimated on previous iterations to quantize the present. Our approach enablesfast static quantization of gradients and activations while requiring onlyminimal hardware support from the neural network accelerator to keep track ofoutput statistics in an online fashion. It is intended as a drop-in replacementfor estimating quantization ranges and can be used in conjunction with otheradvances in quantized training. We compare our method to existing methods forrange estimation from the quantized training literature and demonstrate itseffectiveness with a range of architectures, including MobileNetV2, on imageclassification benchmarks (Tiny ImageNet & ImageNet).

In-Hindsight Quantization Range Estimation for Quantized Training