Action recognition of humanoid robots based on image processing and multi-sensor data fusion

Accurate recognition of humanoid robot actions is essential for autonomous decision-making and adaptive control in dynamic environments. This study proposes an intelligent framework—Image Processing Methods with Multi-Sensor Data Fusion (IPM-MSDF)—to enhance the precision and reliability of interpreting humanoid behavior. Visual data from RGB and depth cameras is combined with proprioceptive inputs from inertial, force, and joint sensors. Spatial features are extracted using Convolutional Neural Networks (CNNs), while Long Short-Term Memory (LSTM) networks model temporal dynamics. Kalman filtering and deep fusion strategies integrate spatial-temporal cues, reducing noise and improving robustness. Experimental evaluation using a proprietary humanoid robot dataset demonstrates 97.5% recognition accuracy, 75 FPS real-time processing, and 90% generalization in unseen environments. The system also achieves 93% computational efficiency and 87% cross-platform adaptability, confirming its scalability and practical viability. These results highlight the potential of multi-modal perception and fusion to advance responsive, context-aware humanoid robotics for industrial, healthcare, and service applications. The framework’s real-time performance makes it particularly suitable for practical deployment in medical settings, such as robotic assistants aiding patient care, and industrial environments, including collaborative manufacturing or hazardous task automation.