English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Vision-Language models (VLMs) pre-trained on large corpora have demonstratednotable success across a range of downstream tasks. In light of the rapidlyincreasing size of pre-trained VLMs, parameter-efficient transfer learning(PETL) has garnered attention as a viable alternative to full fine-tuning. Onesuch approach is the adapter, which introduces a few trainable parameters intothe pre-trained models while preserving the original parameters duringadaptation. In this paper, we present a novel modeling framework that recastsadapter tuning after attention as a graph message passing process on attentiongraphs, where the projected query and value features and attention matrixconstitute the node features and the graph adjacency matrix, respectively.Within this framework, tuning adapters in VLMs necessitates handlingheterophilic graphs, owing to the disparity between the projected query andvalue space. To address this challenge, we propose a new adapter architecture,$p$-adapter, which employs $p$-Laplacian message passing in Graph NeuralNetworks (GNNs). Specifically, the attention weights are re-normalized based onthe features, and the features are then aggregated using the calibratedattention matrix, enabling the dynamic exploitation of information with varyingfrequencies in the heterophilic attention graphs. We conduct extensiveexperiments on different pre-trained VLMs and multi-modal tasks, includingvisual question answering, visual entailment, and image captioning. Theexperimental results validate our method's significant superiority over otherPETL methods.

p-Laplacian Adaptation for Generative Pre-trained Vision-Language Models