Automatic Generation of Robot Actions for Collaborative Tasks from Speech

Robots have the potential to assist people in daily tasks, such as cooking a meal. Communicating with the robots verbally and in an unstructured way is important, as spoken language is the main form of communication for humans. This paper proposes a novel framework that automatically generates robot actions from unstructured speech. The proposed frame-work was evaluated by collecting data from 15 participants preparing their meals while seating on a chair in a randomly disrupted environment. The system can identify and respond to a task sequence while the user may be engaged in unrelated conversations, even if the user's speech might be unstructured and grammatically incorrect. The accuracy of the proposed system is 98.6%, which is a very promising finding.