English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study the understanding of embodied reference: One agent uses bothlanguage and gesture to refer to an object to another agent in a sharedphysical environment. Of note, this new visual task requires understandingmultimodal cues with perspective-taking to identify which object is beingreferred to. To tackle this problem, we introduce YouRefIt, a new crowd-sourceddataset of embodied reference collected in various physical scenes; the datasetcontains 4,195 unique reference clips in 432 indoor scenes. To the best of ourknowledge, this is the first embodied reference dataset that allows us to studyreferring expressions in daily physical scenes to understand referentialbehavior, human communication, and human-robot interaction. We further devisetwo benchmarks for image-based and video-based embodied referenceunderstanding. Comprehensive baselines and extensive experiments provide thevery first result of machine perception on how the referring expressions andgestures affect the embodied reference understanding. Our results provideessential evidence that gestural cues are as critical as language cues inunderstanding the embodied reference.

YouRefIt: Embodied Reference Understanding with Language and Gesture