Answers are grounded in this paper and cite their sources.
Robotic task execution is susceptible to errors due to uncertainties in environmental conditions and physical systems, often resulting in undesirable or potentially hazardous outcomes. Enhancing robot performance necessitates the ability to enable robots to comprehend and rectify abnormal executions. However, contemporary robots possess limited cognitive capabilities for environment comprehension and task reasoning, rendering the self-diagnosis and correction of anomalies a formidable challenge. To address this issue, we present a novel approach known as “human-to-robot attention transfer” (H2R-AT) designed to facilitate human assistance in diagnosing robot abnormalities. H2R-AT leverages a novel stacked neural networks model and an attention mechanism to transfer attention from humans to robots, aiding in the diagnosis of abnormalities. In this process, humans verbally express concerns regarding abnormal robot executions, enabling the robot to discern the nature and location of these concerns, thereby facilitating corrective actions. To evaluate the efficacy of H2R-AT, we devised two representative task scenarios: “serving water for a human in a kitchen” and “picking up a defective gear in a factory,” both involving abnormal behaviors, within the simulation platform craihri. We recruited 252 volunteers who provided approximately 12,000 verbal reminders for learning and testing the attention transfer model H2R-AT. Achieving an accuracy of 73.68% in attention transfer and 90.75% in correctly implementing suggested corrections, the model demonstrated an overall accuracy of 66.86% in preventing robot execution failures. These results affirm the effectiveness of H2R-AT in averting robot failures during task execution.
No takes yet. Share an insight, caveat, or question.
Guo et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: