English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Generalisation to unseen contexts remains a challenge for embodied navigationagents. In the context of semantic audio-visual navigation (SAVi) tasks, thenotion of generalisation should include both generalising to unseen indoorvisual scenes as well as generalising to unheard sounding objects. However,previous SAVi task definitions do not include evaluation conditions on trulynovel sounding objects, resorting instead to evaluating agents on unheard soundclips of known objects; meanwhile, previous SAVi methods do not includeexplicit mechanisms for incorporating domain knowledge about object and regionsemantics. These weaknesses limit the development and assessment of models'abilities to generalise their learned experience. In this work, we introducethe use of knowledge-driven scene priors in the semantic audio-visual embodiednavigation task: we combine semantic information from our novel knowledge graphthat encodes object-region relations, spatial knowledge from dual Graph EncoderNetworks, and background knowledge from a series of pre-training tasks -- allwithin a reinforcement learning framework for audio-visual navigation. We alsodefine a new audio-visual navigation sub-task, where agents are evaluated onnovel sounding objects, as opposed to unheard clips of known objects. We showimprovements over strong baselines in generalisation to unseen regions andnovel sounding objects, within the Habitat-Matterport3D simulation environment,under the SoundSpaces task.

Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation