English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning common subspace is prevalent way in cross-modal retrieval to solvethe problem of data from different modalities having inconsistent distributionsand representations that cannot be directly compared. Previous cross-modalretrieval methods focus on projecting the cross-modal data into a common spaceby learning the correlation between them to bridge the modality gap. However,the rich semantic information in the video and the heterogeneous nature ofaudio-visual data leads to more serious heterogeneous gaps intuitively, whichmay lead to the loss of key semantic content of video with single clue by theprevious methods when eliminating the modality gap, while the semantics of thecategories may undermine the properties of the original features. In this work,we aim to learn effective audio-visual representations to support audio-visualcross-modal retrieval (AVCMR). We propose a novel model that maps audio-visualmodalities into two distinct shared latent subspaces: explicit and implicitshared spaces. In particular, the explicit shared space is used to optimizepairwise correlations, where learned representations across modalities capturethe commonalities of audio-visual pairs and reduce the modality gap. Theimplicit shared space is used to preserve the distinctive features betweenmodalities by maintaining the discrimination of audio/video patterns fromdifferent semantic categories. Finally, the fusion of the features learned fromthe two latent subspaces is used for the similarity computation of the AVCMRtask. The comprehensive experimental results on two audio-visual datasetsdemonstrate that our proposed model for using two different latent subspacesfor audio-visual cross-modal learning is effective and significantlyoutperforms the state-of-the-art cross-modal models that learn features from asingle subspace.

Learning Explicit and Implicit Latent Common Spaces for Audio-Visual Cross-Modal Retrieval