English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent works have shown that Large Language Models (LLMs) can promotegrounding instructions to robotic task planning. Despite the progress, mostexisting works focused on utilizing raw images to help LLMs understandenvironmental information, which not only limits the observation scope but alsotypically requires massive multimodal data collection and large-scale models.In this paper, we propose a novel approach called Graph-based RoboticInstruction Decomposer (GRID), leverages scene graph instead of image toperceive global scene information and continuously plans subtask in each stagefor a given instruction. Our method encodes object attributes and relationshipsin graphs through an LLM and Graph Attention Networks, integrating instructionfeatures to predict subtasks consisting of pre-defined robot actions and targetobjects in the scene graph. This strategy enables robots to acquire semanticknowledge widely observed in the environment from the scene graph. To train andevaluate GRID, we build a dataset construction pipeline to generate syntheticdatasets in graph-based robotic task planning. Experiments have shown that ourmethod outperforms GPT-4 by over 25.4% in subtask accuracy and 43.6% in taskaccuracy. Experiments conducted on datasets of unseen scenes and scenes withdifferent numbers of objects showed that the task accuracy of GRID declined byat most 3.8%, which demonstrates its good cross-scene generalization ability.We validate our method in both physical simulation and the real world.

GRID: Scene-Graph-based Instruction-driven Robotic Task Planning