Real-Time Video-Based Human Action Recognition on Embedded Platforms

Advances in computer vision and deep learning have made video-based Human Action Recognition (HAR) increasingly feasible. However, running HAR on live video streams encounters significant delays on embedded platforms due to computational demands. This work addresses real-time HAR performance challenges through four key contributions: 1) an experimental study identifying standard Optical Flow (OF) extraction as the primary latency bottleneck in a state-of-the-art HAR pipeline, 2) an analysis of the latency-accuracy trade-off between traditional and deep learning-based OF methods, underscoring the need for an efficient motion feature extractor with minimal impact on accuracy, 3) the design of Integrated Motion Feature Extractor (IMFE) , a novel unified neural network architecture that substantially reduces motion feature extraction latency, and 4) the development of RT-HARE , a real-time HAR system optimized for embedded platforms. Experiments on three benchmark datasets of various characteristics using the Nvidia Jetson Xavier NX platform demonstrate that RT-HARE achieves real-time HAR with lower and more stable latency, reduced power consumption, and a smaller memory footprint while maintaining recognition accuracy comparable to more complex server-based HAR models.