English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Robots operating in human-centric environments require the integration ofvisual grounding and grasping capabilities to effectively manipulate objectsbased on user instructions. This work focuses on the task of referring graspsynthesis, which predicts a grasp pose for an object referred through naturallanguage in cluttered scenes. Existing approaches often employ multi-stagepipelines that first segment the referred object and then propose a suitablegrasp, and are evaluated in private datasets or simulators that do not capturethe complexity of natural indoor scenes. To address these limitations, wedevelop a challenging benchmark based on cluttered indoor scenes from OCIDdataset, for which we generate referring expressions and connect them with4-DoF grasp poses. Further, we propose a novel end-to-end model (CROG) thatleverages the visual grounding capabilities of CLIP to learn grasp synthesisdirectly from image-text pairs. Our results show that vanilla integration ofCLIP with pretrained models transfers poorly in our challenging benchmark,while CROG achieves significant improvements both in terms of grounding andgrasping. Extensive robot experiments in both simulation and hardwaredemonstrate the effectiveness of our approach in challenging interactive objectgrasping scenarios that include clutter.

Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter