English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Embodied AI models often employ off the shelf vision backbones like CLIP toencode their visual observations. Although such general purpose representationsencode rich syntactic and semantic information about the scene, much of thisinformation is often irrelevant to the specific task at hand. This introducesnoise within the learning process and distracts the agent's focus fromtask-relevant visual cues. Inspired by selective attention in humans-theprocess through which people filter their perception based on theirexperiences, knowledge, and the task at hand-we introduce a parameter-efficientapproach to filter visual stimuli for embodied AI. Our approach induces atask-conditioned bottleneck using a small learnable codebook module. Thiscodebook is trained jointly to optimize task reward and acts as atask-conditioned selective filter over the visual observation. Our experimentsshowcase state-of-the-art performance for object goal navigation and objectdisplacement across 5 benchmarks, ProcTHOR, ArchitecTHOR, RoboTHOR, AI2-iTHOR,and ManipulaTHOR. The filtered representations produced by the codebook arealso able generalize better and converge faster when adapted to othersimulation environments such as Habitat. Our qualitative analyses show thatagents explore their environments more effectively and their representationsretain task-relevant information like target object recognition while ignoringsuperfluous information about other objects. Code and pretrained models areavailable at our project website: https://embodied-codebook.github.io.

Selective Visual Representations Improve Convergence and Generalization for Embodied AI