English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision-and-language navigation (VLN) is a multimodal task where an agentfollows natural language instructions and navigates in visual environments.Multiple setups have been proposed, and researchers apply new modelarchitectures or training techniques to boost navigation performance. However,there still exist non-negligible gaps between machines' performance and humanbenchmarks. Moreover, the agents' inner mechanisms for navigation decisionsremain unclear. To the best of our knowledge, how the agents perceive themultimodal input is under-studied and needs investigation. In this work, weconduct a series of diagnostic experiments to unveil agents' focus duringnavigation. Results show that indoor navigation agents refer to both object anddirection tokens when making decisions. In contrast, outdoor navigation agentsheavily rely on direction tokens and poorly understand the object tokens.Transformer-based agents acquire a better cross-modal understanding of objectsand display strong numerical reasoning ability than non-Transformer-basedagents. When it comes to vision-and-language alignments, many models claim thatthey can align object tokens with specific visual targets. We find unbalancedattention on the vision and text input and doubt the reliability of suchcross-modal alignments.

Diagnosing Vision-and-Language Navigation: What Really Matters