English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Large language models (LLMs) have revolutionized natural language processingtasks. However, their practical deployment is hindered by their immense memoryand computation requirements. Although recent post-training quantization (PTQ)methods are effective in reducing memory footprint and improving thecomputational efficiency of LLM, they hand-craft quantization parameters, whichleads to low performance and fails to deal with extremely low-bit quantization.To tackle this issue, we introduce an Omnidirectionally calibrated Quantization(OmniQuant) technique for LLMs, which achieves good performance in diversequantization settings while maintaining the computational efficiency of PTQ byefficiently optimizing various quantization parameters. OmniQuant comprises twoinnovative components including Learnable Weight Clipping (LWC) and LearnableEquivalent Transformation (LET). LWC modulates the extreme values of weights byoptimizing the clipping threshold. Meanwhile, LET tackles activation outliersby shifting the challenge of quantization from activations to weights through alearnable equivalent transformation. Operating within a differentiableframework using block-wise error minimization, OmniQuant can optimize thequantization process efficiently for both weight-only and weight-activationquantization. For instance, the LLaMA-2 model family with the size of 7-70B canbe processed with OmniQuant on a single A100-40G GPU within 1-16 hours using128 samples. Extensive experiments validate OmniQuant's superior performanceacross diverse quantization configurations such as W4A4, W6A6, W4A16, W3A16,and W2A16. Additionally, OmniQuant demonstrates effectiveness ininstruction-tuned models and delivers notable improvements in inference speedand memory reduction on real devices. Codes and models are available at\url{https://github.com/OpenGVLab/OmniQuant}.

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models