English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Sound source localization in visual scenes aims to localize objects emittingthe sound in a given image. Recent works showing impressive localizationperformance typically rely on the contrastive learning framework. However, therandom sampling of negatives, as commonly adopted in these methods, can resultin misalignment between audio and visual features and thus inducing ambiguityin localization. In this paper, instead of following previous literature, wepropose Self-Supervised Predictive Learning (SSPL), a negative-free method forsound localization via explicit positive mining. Specifically, we first devisea three-stream network to elegantly associate sound source with two augmentedviews of one corresponding video frame, leading to semantically coherentsimilarities between audio and visual features. Second, we introduce a novelpredictive coding module for audio-visual feature alignment. Such a moduleassists SSPL to focus on target objects in a progressive manner and effectivelylowers the positive-pair learning difficulty. Experiments show surprisingresults that SSPL outperforms the state-of-the-art approach on two standardsound localization benchmarks. In particular, SSPL achieves significantimprovements of 8.6% cIoU and 3.4% AUC on SoundNet-Flickr compared to theprevious best. Code is available at: https://github.com/zjsong/SSPL.

Self-Supervised Predictive Learning: A Negative-Free Method for Sound Source Localization in Visual Scenes