English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Audio-visual navigation combines sight and hearing to navigate to asound-emitting source in an unmapped environment. While recent approaches havedemonstrated the benefits of audio input to detect and find the goal, theyfocus on clean and static sound sources and struggle to generalize to unheardsounds. In this work, we propose the novel dynamic audio-visual navigationbenchmark which requires catching a moving sound source in an environment withnoisy and distracting sounds, posing a range of new challenges. We introduce areinforcement learning approach that learns a robust navigation policy forthese complex settings. To achieve this, we propose an architecture that fusesaudio-visual information in the spatial feature space to learn correlations ofgeometric information inherent in both local maps and audio signals. Wedemonstrate that our approach consistently outperforms the currentstate-of-the-art by a large margin across all tasks of moving sounds, unheardsounds, and noisy environments, on two challenging 3D scanned real-worldenvironments, namely Matterport3D and Replica. The benchmark is available athttp://dav-nav.cs.uni-freiburg.de.

Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Sounds