Vision-based touchless gesture recognition for clinical HCI: deployment challenges, trade-offs, and future directions

Touchless gesture recognition by vision represents a promising sterile and non-contact human-computer interface (HCI) for clinical settings, where sterility and continuous workflow are paramount, yet its practical deployment remains substantially unrealised. This systematic review evaluates the deployment of RGB, RGB-D, and skeletal-vision systems across operating rooms, intensive care units, and rehabilitation settings. Following PRISMA guidelines, literature published between 2018 and 2026 was analysed, with emphasis on clinically deployed and near-clinical systems rather than purely algorithmic benchmarks. Recent deep learning methods, such as convolutional neural networks (CNNs), long short-term memory (LSTM) networks, transformers, and multimodal fusion models, have been shown to achieve laboratory accuracy of more than 95%. However, real-world performance degrades by 1-23 percentage points due to occlusion, high-intensity surgical lighting (500–2000 lux), background complexity, and hardware limitations. This review compares and contrasts the clinical scenarios and identifies the major accuracy-latency trade-offs and failure modes of operation. It suggests the use of a three-tier deployment model where lightweight techniques are considered for safety-critical commands only after task-specific validation, hybrid CNN-LSTM models are used in regular interactions, and computationally costly models are used in offline or experimental applications. The main issues are the bias of the dataset, ambiguity of the gestures, and the lack of interoperability with proprietary electronic health record systems. Future research directions are domain adaptation, federated learning, and edge-optimised multimodal systems.