A Hybrid Architecture for Vision-Based Virtual Mouse Systems: Normalized Gesture Classification and Rule-Based OS Event Dispatching

Vision-based human-computer interaction (HCI) interfaces frequently suffer from operational in-stability and actuation ambiguity, including high-frequency cursor jitter, unintentional multi-click triggers, and unintended context-menu dismissals caused by hardware-dependent frame rate vari-ations and involuntary physiological hand tremors. This paper presents an empirically evaluated hybrid architecture for real-time virtual mouse control that bridges probabilistic computer vision inference with rule-based operating system (OS) event-dispatching constraints. The proposed pipeline decouples spatial landmark kinematics from intentional state recognition: 21 three-dimensional hand landmarks are extracted via MediaPipe Tasks in continuous video mode, stabilized using a temporal wrist-anchoring state machine to reduce unintended target-hand switching. Retained 2D image-plane components undergo translation normalization, approximate uniform-scale normalization, and handedness canonicalization via horizontal reflection. Evaluated on a curated dataset of 10,008 em-pirical instances collected across five human subjects and six operational classes (Move, Left_Click, Right_Click, Drag, Scroll, and Neutral), an RBF-kernel Support Vector Machine (SVM) pipeline achieves 98.10% sample-level test accuracy under a stratified random split with an isolated inference latency of 0.717 ms (averaging 0.72 ms under steady-state daemon execution), paired with rapid model training time (0.68 s) enabling fast local model retraining. Crucially, the frozen classifier pipeline (including fitted feature scaling) was independently evaluated in a zero-shot evaluation on three unseen human subjects (766 sequential test frames), achieving an overall cross-subject accuracy of 88.12% and macro F1-score of 0.88 without subject-specific retraining. Complementary live interactive pilot sessions across 10 human participants demonstrated operational usability across the primary interaction modes under the tested desktop conditions. Downstream, an FPS-decoupled, wall-clock rate-limiting mechanism (0.25 s refractory gate) combined with coordinate freezing, an event-latched state machine, and two-frame drag hysteresis regulates actuation frequency, retains context menus, and maintains stable logged actuation coordinates across exploratory live ablation trials under the tested conditions. Furthermore, a thread-safe circular telemetry buffer isolates vision processing from GUI rendering, delivering continuous millisecond-resolution audit tracing for both dispatched actions and suppressed event candidates, with an average post-detection pipeline overhead of approximately 1.22 ms.