English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision transformers (ViTs) have achieved impressive results on variouscomputer vision tasks in the last several years. In this work, we study thecapability of frozen ViTs, pretrained only on visual data, to generalize toaudio-visual data without finetuning any of its original parameters. To do so,we propose a latent audio-visual hybrid (LAVISH) adapter that adapts pretrainedViTs to audio-visual tasks by injecting a small number of trainable parametersinto every layer of a frozen ViT. To efficiently fuse visual and audio cues,our LAVISH adapter uses a small set of latent tokens, which form an attentionbottleneck, thus, eliminating the quadratic cost of standard cross-attention.Compared to the existing modality-specific audio-visual methods, our approachachieves competitive or even better performance on various audio-visual taskswhile using fewer tunable parameters and without relying on costly audiopretraining or external audio encoders. Our code is available athttps://genjib.github.io/project_page/LAVISH/

Vision Transformers are Parameter-Efficient Audio-Visual Learners