English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Post-training quantization~(PTQ) of transformer language models facessignificant challenges due to the existence of detrimental outliers inactivations. We observe that these outliers are concentrated in specificchannels and are asymmetric across channels. To address this issue, we proposethe Outlier Suppression+~(OS+) framework, which contains the channel-wiseshifting for asymmetry and channel-wise scaling for concentration. We show thatthese operations can be seamlessly migrated into subsequent modules whilemaintaining equivalence. Second, we propose a fast and stable scheme tocalculate effective shifting and scaling values. The channel-wise shiftingaligns the center of each channel for removal of outlier asymmetry. Thechannel-wise scaling quantitatively evaluates changes brought by migration andquantization for better quantization burden balance. We validate our OS+ underboth standard and fine-grained quantization settings with models includingBERT, OPT, BLOOM, BLOOMZ, and LLaMA. Comprehensive results across various tasksdemonstrate the superiority of our approach. Especially, with standardquantization, OS+ can achieve near-floating-point performance on both smallmodels and large language models on 8-bit and 6-bit. Besides, we establish anew state-of-the-art for 4-bit BERT with 15.5\% improvement. Our code isavailable at \url{https://github.com/ModelTC/Outlier_Suppression_Plus}.

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling