English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Current state-of-the-art results in computer vision depend in part onfine-tuning large pre-trained vision models. However, with the exponentialgrowth of model sizes, the conventional full fine-tuning, which needs to storea individual network copy for each tasks, leads to increasingly huge storageand transmission overhead. Adapter-based Parameter-Efficient Tuning (PET)methods address this challenge by tuning lightweight adapters inserted into thefrozen pre-trained models. In this paper, we investigate how to make adapterseven more efficient, reaching a new minimum size required to store atask-specific fine-tuned network. Inspired by the observation that theparameters of adapters converge at flat local minima, we find that adapters areresistant to noise in parameter space, which means they are also resistant tolow numerical precision. To train low-precision adapters, we propose acomputational-efficient quantization method which minimizes the quantizationerror. Through extensive experiments, we find that low-precision adaptersexhibit minimal performance degradation, and even 1-bit precision is sufficientfor adapters. The experimental results demonstrate that 1-bit adaptersoutperform all other PET methods on both the VTAB-1K benchmark and few-shotFGVC tasks, while requiring the smallest storage size. Our findings show, forthe first time, the significant potential of quantization techniques in PET,providing a general solution to enhance the parameter efficiency ofadapter-based PET methods. Code: https://github.com/JieShibo/PETL-ViT

Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy