English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Remarkable progress has been made in recent years in the fields of vision,language, and robotics. We now have vision models capable of recognizingobjects based on language queries, navigation systems that can effectivelycontrol mobile systems, and grasping models that can handle a wide range ofobjects. Despite these advancements, general-purpose applications of roboticsstill lag behind, even though they rely on these fundamental capabilities ofrecognition, navigation, and grasping. In this paper, we adopt a systems-firstapproach to develop a new Open Knowledge-based robotics framework calledOK-Robot. By combining Vision-Language Models (VLMs) for object detection,navigation primitives for movement, and grasping primitives for objectmanipulation, OK-Robot offers a integrated solution for pick-and-dropoperations without requiring any training. To evaluate its performance, we runOK-Robot in 10 real-world home environments. The results demonstrate thatOK-Robot achieves a 58.5% success rate in open-ended pick-and-drop tasks,representing a new state-of-the-art in Open Vocabulary Mobile Manipulation(OVMM) with nearly 1.8x the performance of prior work. On cleaner, unclutteredenvironments, OK-Robot's performance increases to 82%. However, the mostimportant insight gained from OK-Robot is the critical role of nuanced detailswhen combining Open Knowledge systems like VLMs with robotic modules. Videos ofour experiments are available on our website: https://ok-robot.github.io

OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics