English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Audiovisual scenes are pervasive in our daily life. It is commonplace forhumans to discriminatively localize different sounding objects but quitechallenging for machines to achieve class-aware sounding objects localizationwithout category annotations, i.e., localizing the sounding object andrecognizing its category. To address this problem, we propose a two-stagestep-by-step learning framework to localize and recognize sounding objects incomplex audiovisual scenarios using only the correspondence between audio andvision. First, we propose to determine the sounding area via coarse-grainedaudiovisual correspondence in the single source cases. Then visual features inthe sounding area are leveraged as candidate object representations toestablish a category-representation object dictionary for expressive visualcharacter extraction. We generate class-aware object localization maps incocktail-party scenarios and use audiovisual correspondence to suppress silentareas by referring to this dictionary. Finally, we employ category-levelaudiovisual consistency as the supervision to achieve fine-grained audio andsounding object distribution alignment. Experiments on both realistic andsynthesized videos show that our model is superior in localizing andrecognizing objects as well as filtering out silent ones. We also transfer thelearned audiovisual network into the unsupervised object detection task,obtaining reasonable performance.

Class-aware Sounding Objects Localization via Audiovisual Correspondence