ABSTRACT This study presents a novel tri‐manual interaction framework that enables users to control two physical hands via VR controllers and a virtual robotic arm through a hybrid brain‐computer interface (32‐channel EEG and eye‐tracking). The virtual robotic arm, implemented as a 6‐DOF industrial manipulator in Unity, is controlled through simplified BCI commands: forward/backward movement along the z‐axis based on motor imagery strength, with automatic grasping triggered by sustained attention thresholds. Twenty‐five participants completed 30 trials, each following a 60‐s protocol with five phases: rest, target presentation, preparation, execution, and feedback. Results demonstrated that the hybrid BCI system achieved superior performance compared to EEG‐only control: 8.5% improvement in task success rate, 34.5% increase in positioning accuracy, and 46.2% reduction in cognitive load. The CNN‐LSTM architecture achieved 86.3% motor imagery classification accuracy. Learning effects were observed within trials, with performance plateauing after 3.8 ± 0.7 attempts. Time‐frequency analysis revealed hierarchical neural coordination mechanisms underlying the tri‐manual control. This research validates the feasibility of augmented manipulation in virtual environments, establishing a foundation for advanced human‐robot collaboration in animation and VR applications.
Teng et al. (2025) studied this question.