English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large DNNs with mixed-precision quantization can achieve ultra-highcompression while retaining high classification performance. However, becauseof the challenges in finding an accurate metric that can guide the optimizationprocess, these methods either sacrifice significant performance compared to the32-bit floating-point (FP-32) baseline or rely on a compute-expensive,iterative training policy that requires the availability of a pre-trainedbaseline. To address this issue, this paper presents BMPQ, a training methodthat uses bit gradients to analyze layer sensitivities and yieldmixed-precision quantized models. BMPQ requires a single training iteration butdoes not need a pre-trained baseline. It uses an integer linear program (ILP)to dynamically adjust the precision of layers during training, subject to afixed hardware budget. To evaluate the efficacy of BMPQ, we conduct extensiveexperiments with VGG16 and ResNet18 on CIFAR-10, CIFAR-100, and Tiny-ImageNetdatasets. Compared to the baseline FP-32 models, BMPQ can yield models thathave 15.4x fewer parameter bits with a negligible drop in accuracy. Compared tothe SOTA "during training", mixed-precision training scheme, our models are2.1x, 2.2x, and 2.9x smaller, on CIFAR-10, CIFAR-100, and Tiny-ImageNet,respectively, with an improved accuracy of up to 14.54%.

BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratch