Programming cobots by voice: A human-centered, web-based approach

We present a new voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces (HPIs), which can be used by people with little or no programming experience. The framework follows a meta-programming approach by enabling users to program cobots by voice in addition to using a mouse, tablet or keyboard. Upon a voice instruction (such as move, pick, release, etc.), the framework automates the manual tasks required to manipulate the vendor-provided HPI. The main advantages of this approach are simplified, guided programming, which only requires the knowledge of 5–10 voice instructions; increased programming speed by up to 46% compared to the manual approach; and the possibility of sharing programs as videos. The open-source framework was evaluated using two application scenarios.