English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Previous works on scene classification are mainly based on audio or visualsignals, while humans perceive the environmental scenes through multiplesenses. Recent studies on audio-visual scene classification separatelyfine-tune the largescale audio and image pre-trained models on the targetdataset, then either fuse the intermediate representations of the audio modeland the visual model, or fuse the coarse-grained decision of both models at theclip level. Such methods ignore the detailed audio events and visual objects inaudio-visual scenes (AVS), while humans often identify different scenes throughaudio events and visual objects within and the congruence between them. Toexploit the fine-grained information of audio events and visual objects in AVS,and coordinate the implicit relationship between audio events and visualobjects, this paper proposes a multibranch model equipped with contrastiveevent-object alignment (CEOA) and semantic-based fusion (SF) for AVSC. CEOAaims to align the learned embeddings of audio events and visual objects bycomparing the difference between audio-visual event-object pairs. Then, visualobjects associated with certain audio events and vice versa are accentuated bycross-attention and undergo SF for semantic-level fusion. Experiments showthat: 1) the proposed AVSC model equipped with CEOA and SF outperforms theresults of audio-only and visual-only models, i.e., the audio-visual resultsare better than the results from a single modality. 2) CEOA aligns theembeddings of audio events and related visual objects on a fine-grained level,and the SF effectively integrates both; 3) Compared with other large-scaleintegrated systems, the proposed model shows competitive performance, evenwithout using additional datasets and data augmentation tricks.

Audio-visual scene classification via contrastive event-object alignment and semantic-based fusion