English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) and vision language models (VLMs) demonstrateexcellent performance on a wide range of tasks by scaling up parameter countsfrom O(10^9) to O(10^{12}) levels and further beyond. These large scales makeit impossible to adapt and deploy fully specialized models given a task ofinterest. Parameter-efficient fine-tuning (PEFT) emerges as a promisingdirection to tackle the adaptation and serving challenges for such largemodels. We categorize PEFT techniques into two types: intrusive andnon-intrusive. Intrusive PEFT techniques directly change a model's internalarchitecture. Though more flexible, they introduce significant complexities fortraining and serving. Non-intrusive PEFT techniques leave the internalarchitecture unchanged and only adapt model-external parameters, such asembeddings for input. In this work, we describe AdaLink as a non-intrusive PEFTtechnique that achieves competitive performance compared to SoTA intrusive PEFT(LoRA) and full model fine-tuning (FT) on various tasks. We evaluate using bothtext-only and multimodal tasks, with experiments that account for bothparameter-count scaling and training regime (with and without instructiontuning).

Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling