English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a memory-efficient finetuning algorithm for large language models(LLMs) that supports finetuning LLMs with 65B parameters in 3-bit or 4-bitprecision on as little as one 48GB GPU. Our method, modular low-rank adaptation(ModuLoRA), integrates any user-specified weight quantizer with finetuning vialow-rank adapters (LoRAs). Our approach relies on a simplequantization-agnostic backward pass that adaptively materializes low-precisionLLM weights from a custom black-box quantization module. This approach enablesfinetuning 3-bit LLMs for the first time--leveraging state-of-the-art 3-bitOPTQ quantization often outperforms finetuning that relies on lesssophisticated 4-bit and 8-bit methods. In our experiments, ModuLoRA attainscompetitive performance on text classification, natural language infernece, andinstruction following tasks using significantly less memory than existingapproaches, and we also surpass the state-of-the-art ROUGE score on a popularsummarization task. We release ModuLoRA together with a series of low-precisionmodels--including the first family of 3-bit instruction following AlpacaLLMs--as part of LLMTOOLS, a user-friendly library for quantizing, running, andfinetuning LLMs on consumer GPUs.

ModuLoRA: Finetuning 3-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers