Answers are grounded in this paper and cite their sources.
Advances in large language models (LLMs) have accelerated the real-world deployment of embodied intelligent robots, supporting intuitive human–robot interaction without specialized robotics expertise. However, few systematic assessments examine how well LLMs translate natural-language route instructions into executable robot command sequences, particularly for non-English instructions and across deployment-relevant model classes. In this paper, a Chinese route-instruction corpus was constructed, which comprised 212 instructions collected from six native speakers across 50 procedurally generated town maps. Each was annotated with a ground-truth command sequence verified by simulation to reach its goal, and a 100% performance ceiling was established. Using this corpus, 12 LLMs spanning three capability tiers—cloud proprietary models, mid-sized open-weight models, and small open-weight models—were evaluated under three system prompts differing only in the number of worked examples (zero, one, and three). Open-weight models in the 3–9B range reached at most 9.2% of goals, while proprietary cloud models reached 80.0%, a substantial gap that increasing the number of worked examples did not measurably narrow. A paired analysis across the three prompt conditions detected no reliable effect of demonstration count: pooled path_reach rose modestly from 25.0% under zero-shot prompting to 27.2% with one example and 27.7% with three examples (three versus zero examples: odds ratio: 1.15; 95% CI: 0.98–1.34). None of these differences was significant after multiple-comparison correction (Holm-adjusted p ≥ 0.247). Low-Rank Adaptation (LoRA) fine-tuning on 126 examples raised mid-sized open-weight models to 27.7–35.4%, and the gain spanned both output-format compliance and route competence.
No takes yet. Share an insight, caveat, or question.
Yang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: