Key points are not available for this paper at this time.
We present a framework for building interactive, real-time, natural language-instructable robots in the real world, and we open source related assets (dataset, environment, benchmark, and policies). Trained with behavioral cloning on a dataset of hundreds of thousands of language-annotated trajectories, a produced policy can proficiently execute an order of magnitude more commands than previous works: specifically we estimate a 93.5% success rate on a set of 87,000 unique natural language strings specifying raw end-to-end visuolinguo-motor skills in the real world. We find that the same policy is capable of being guided by a human via real-time language to address a wide range of precise long-horizon rearrangement goals, e.g. “ make a smiley face out of blocks ”. The dataset we release comprises nearly 600,000 language-labeled trajectories, an order of magnitude larger than prior available datasets. We hope the demonstrated results and associated assets enable further advancement of helpful, capable, natural-language-interactable robots. See videos at https://interactive-language.github.io .
Building similarity graph...
Analyzing shared references across papers
Loading...
Corey Lynch
Google (United States)
Ayzaan Wahid
Google (United States)
Jonathan Tompson
Supélec
IEEE Robotics and Automation Letters
Google (United States)
Building similarity graph...
Analyzing shared references across papers
Loading...
Lynch et al. (Mon,) studied this question.
synapsesocial.com/papers/6a0c7e0bf84e7d4200885402 — DOI: https://doi.org/10.1109/lra.2023.3295255