Design and Development of an AI-Based Voice-Controlled Robot for Intelligent Human-Robot Interaction

Most low-cost voice-controlled robots accept only a handful of fixed words such as “forward” and “stop”, and depend on a smartphone app for speech recognition. This paper presents the design and development of an AI-based voice-controlled mobile robot that holds natural spoken conversations and turns free-form requests into safe movements. The robot is built around an ESP32 microcontroller with an INMP441 I²S MEMS microphone, a MAX98357A I²S class-D amplifier and speaker, an L298N dual H-bridge driving two DC gear motors with wheels, a servo for head and gesture movements, and a 2S pack of 18650 lithium-ion cells with a main switch. A hybrid architecture is used: a compact keyword-spotting model on the ESP32 detects the wake word and a local “stop” command without any network, while the spoken request is streamed to an AI server where speech-to-text, a large language model and text-to-speech produce a structured JSON action and a spoken reply. A command validator on the ESP32 checks every action against a whitelist, speed limits and a time-out before the motors move, so the language model can never drive the robot outside its safe envelope. A review of 19 related studies shows that earlier robots offered either fixed-vocabulary control on cheap hardware or natural-language interaction on expensive platforms, but rarely both. Design calculations show that a 4-second request needs 128 kB of audio, well within the ESP32's memory, and that the battery pack supports an estimated 13.4 h of mixed use. An experimental protocol with metrics for recognition accuracy, latency, safety response and user experience is provided for prototype validation.