Interactive task planning through natural language

We describe a learning user interface (LUI) for robot task planning and programming. A robot operator interacts with LUI with commands in a natural-language-like form, and teaches LUI new commands necessary to perform assigned tasks using the programming by demonstration paradigm. LUI has a basic set of commands for moving a robot. When a new form of a command is given along with a demonstration of how to execute the command in terms of a sequence of known commands, LUI abstracts and stores the demonstration sequence. The abstraction involves qualitative reasoning about spatial relationships of the robot and objects involved during the demonstration. LUI allows robot operators to teach and subsequently issue high-level, natural-language-like motion commands rather than precise numeric commands involving coordinate specifications. This capability can speed up robotic operations in many applications in which similar tasks must be carried out repetitively.