English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this study, we focus on multimodal language understanding for fetchinginstructions in the domestic service robots context. This task consists ofpredicting a target object, as instructed by the user, given an image and anunstructured sentence, such as "Bring me the yellow box (from the woodencabinet)." This is challenging because of the ambiguity of natural language,i.e., the relevant information may be missing or there might be severalcandidates. To solve such a task, we propose the multimodal target-sourceclassifier model with attention branches (MTCM-AB), which is an extension ofthe MTCM. Our methodology uses the attention branch network (ABN) to develop amultimodal attention mechanism based on linguistic and visual inputs.Experimental validation using a standard dataset showed that the MTCM-ABoutperformed both state-of-the-art methods and the MTCM. In particular theMTCM-AB accuracy on average was 90.1% while human performance was 90.3% on thePFN-PIC dataset.

A Multimodal Target-Source Classifier with Attention Branches to Understand Ambiguous Instructions for Fetching Daily Objects