English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a simple approach for memory-efficient adaptation of pretrainedlanguage models. Our approach uses an iterative algorithm to decompose eachpretrained matrix into a high-precision low-rank component and amemory-efficient quantized component. During finetuning, the quantizedcomponent remains fixed and only the low-rank component is updated. We presentan integer linear programming formulation of the quantization component whichenables dynamic configuration of quantization parameters (e.g., bit-width,block size) for each matrix given an overall target memory budget. We furtherexplore a data-aware version of the algorithm which uses an approximation ofthe Fisher information matrix to weight the reconstruction objective duringmatrix decomposition. Experiments on finetuning RoBERTa and LLaMA-2 (7B and70B) demonstrate that our low-rank plus quantized matrix decomposition approach(LQ-LoRA) outperforms strong QLoRA and GPTQ-LoRA baselines and enablesaggressive quantization to sub-3 bits with only minor performance degradations.When finetuned on a language modeling calibration dataset, LQ-LoRA can also beused for model compression; in this setting our 2.75-bit LLaMA-2-70B model(which has 2.85 bits on average when including the low-rank components andrequires 27GB of GPU memory) performs respectably compared to the 16-bitbaseline.

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning