English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Adapting large-scale pretrained models to various downstream tasks viafine-tuning is a standard method in machine learning. Recently,parameter-efficient fine-tuning methods show promise in adapting a pretrainedmodel to different tasks while training only a few parameters. Despite theirsuccess, most existing methods are proposed in Natural Language Processingtasks with language Transformers, and adaptation to Computer Vision tasks withVision Transformers remains under-explored, especially for dense vision tasks.Further, in multi-task settings, individually fine-tuning and storing separatemodels for different tasks is inefficient. In this work, we provide anextensive multi-task parameter-efficient benchmark and examine existingparameter-efficient fine-tuning NLP methods for vision tasks. Our results onfour different dense vision tasks showed that existing methods cannot beefficiently integrated due to the hierarchical nature of the HierarchicalVision Transformers. To overcome this issue, we propose Polyhistor andPolyhistor-Lite, consisting of Decomposed HyperNetworks and Layer-wise ScalingKernels, to share information across different tasks with a few trainableparameters. This leads to favorable performance improvements against existingparameter-efficient methods while using fewer trainable parameters.Specifically, Polyhistor achieves competitive accuracy compared to thestate-of-the-art while only using ~10% of their trainable parameters.Furthermore, our methods show larger performance gains when large networks andmore pretraining data are used.

Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks