Context-Aware Multimodal Voice Command Recognition System for Smart Home Applications.

AbstractThe growing prevalence of smart homes has redefined how users interact with technology, shifting from manual interfaces to intuitive,voice-driven systems . Voice Command Recognition (VCR) is a cornerstone of this transformation, enabling hands-freecontrol overdevices such as lighting systems, entertainment units, security appliances, and climate controllers. Through IoT, voice assistants likeAmazon Alexa, Google Assistant, and Apple Siri can control various smart home devices such as lights, thermostats, and securitycameras.These devices communicate over a network, allowing the voice assistant to send commands and receive real-time feedback.However, real-world conditions such as overlapping speech, background noise, and varying room acoustics pose significant challenges tothe accuracy and reliability of voice-controlled systems. In addition to these, voiceassistants struggle with context management andinterpreting complex commands involving anaphora or ambiguous references, leading to potential errors in command execution. Toaddress these issues, modern smart home systems must not only interpret commands but also isolate them from complex soundmixtures indynamic environments and also effectively tackles the challenges of anaphora in these systems by enhancing context retention anddisambiguation.This work proposes an optimized deep learning framework for source separation and voice command recognition for smarthome systems and improve the understanding for anaphora in these systems. Initially, the raw smart home commands dataset collected from a freely accessible source will be converted into audio clips, which generates audio from text for voice command recognition. Then, preprocessing will be performed using the chained Savitsky-Golay filtering (CSGF) technique and source separation is performed using theconvoluted variational autoencoder (ConvVAE). After source separation, various audio features will be extracted, The extracted features will then fed into the multiscale dilated separable convolutional network (MDSCN) to recognize multiple voice commands for smart homesystems accurately. Then this processed voice commands are fed to a predefined recurrent network to addresses the anaphora challenges insequential commands given. It also addresses the compound anaphora issues by integrating structured rule-based interactions withadvanced AI techniques. This combined approach enhances the system's ability to maintain context and apply actions accurately acrossmultiple devices.This dual-layered approach significantly improves the reliability, precision, and naturalness of smart voice assistants in dynamic acoustic and dialogic environments.