English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Domestic service robots offer a solution to the increasing demand for dailycare and support. A human-in-the-loop approach that combines automation andoperator intervention is considered to be a realistic approach to their use insociety. Therefore, we focus on the task of retrieving target objects fromopen-vocabulary user instructions in a human-in-the-loop setting, which wedefine as the learning-to-rank physical objects (LTRPO) task. For example,given the instruction "Please go to the dining room which has a round table.Pick up the bottle on it," the model is required to output a ranked list oftarget objects that the operator/user can select. In this paper, we proposeMultiRankIt, which is a novel approach for the LTRPO task. MultiRankItintroduces the Crossmodal Noun Phrase Encoder to model the relationship betweenphrases that contain referring expressions and the target bounding box, and theCrossmodal Region Feature Encoder to model the relationship between the targetobject and multiple images of its surrounding contextual environment.Additionally, we built a new dataset for the LTRPO task that consists ofinstructions with complex referring expressions accompanied by real indoorenvironmental images that feature various target objects. We validated ourmodel on the dataset and it outperformed the baseline method in terms of themean reciprocal rank and recall@k. Furthermore, we conducted physicalexperiments in a setting where a domestic service robot retrieved everydayobjects in a standardized domestic environment, based on users' instruction ina human--in--the--loop setting. The experimental results demonstrate that thesuccess rate for object retrieval achieved 80%. Our code is available athttps://github.com/keio-smilab23/MultiRankIt.

Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine