English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Post-training quantization (PTQ) has been gaining popularity for thedeployment of deep neural networks on resource-limited devices since unlikequantization-aware training, neither a full training dataset nor end-to-endtraining is required at all. As PTQ schemes based on reconstructing each layeror block output turn out to be effective to enhance quantized modelperformance, recent works have developed algorithms to devise and learn a newweight-rounding scheme so as to better reconstruct each layer or block output.In this work, we propose a simple yet effective new weight-rounding mechanismfor PTQ, coined FlexRound, based on element-wise division instead of typicalelement-wise addition such that FlexRound enables jointly learning a commonquantization grid size as well as a different scale for each pre-trainedweight. Thanks to the reciprocal rule of derivatives induced by element-wisedivision, FlexRound is inherently able to exploit pre-trained weights whenupdating their corresponding scales, and thus, flexibly quantize pre-trainedweights depending on their magnitudes. We empirically validate the efficacy ofFlexRound on a wide range of models and tasks. To the best of our knowledge,our work is the first to carry out comprehensive experiments on not only imageclassification and natural language understanding but also natural languagegeneration, assuming a per-tensor uniform PTQ setting. Moreover, wedemonstrate, for the first time, that large language models can be efficientlyquantized, with only a negligible impact on performance compared tohalf-precision baselines, achieved by reconstructing the output in ablock-by-block manner.

FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization