English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, post-training quantization (PTQ) has driven much attention toproduce efficient neural networks without long-time retraining. Despite its lowcost, current PTQ works tend to fail under the extremely low-bit setting. Inthis study, we pioneeringly confirm that properly incorporating activationquantization into the PTQ reconstruction benefits the final accuracy. To deeplyunderstand the inherent reason, a theoretical framework is established,indicating that the flatness of the optimized low-bit model on calibration andtest data is crucial. Based on the conclusion, a simple yet effective approachdubbed as QDROP is proposed, which randomly drops the quantization ofactivations during PTQ. Extensive experiments on various tasks includingcomputer vision (image classification, object detection) and natural languageprocessing (text classification and question answering) prove its superiority.With QDROP, the limit of PTQ is pushed to the 2-bit activation for the firsttime and the accuracy boost can be up to 51.49%. Without bells and whistles,QDROP establishes a new state of the art for PTQ. Our code is available athttps://github.com/wimh966/QDrop and has been integrated into MQBench(https://github.com/ModelTC/MQBench)

QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization