English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pruning and quantization form the foundation of model compression for neuralnetworks, enabling efficient inference for large language models (LLMs).Recently, various quantization and pruning techniques have demonstratedstate-of-the-art performance in a post-training setting. They rely uponcalibration data, a small set of unlabeled examples, to generate layeractivations. However, no prior work has systematically investigated how thecalibration data impacts the effectiveness of model compression methods. Inthis paper, we present the first extensive empirical study on the effect ofcalibration data upon LLM performance. We trial a variety of pruning andquantization methods, tasks, models, and datasets. Surprisingly, we findsubstantial variations in downstream task performance, contrasting existingwork that suggests a greater level of robustness to the calibration data.Finally, we make a series of recommendations for the effective use ofcalibration data in LLM quantization and pruning.

How Does Calibration Data Affect the Post-training Pruning and Quantization of Large Language Models?