Real-Time and On-Board Anomalous Command Detection in UAV Operations via Simultaneous UAV-Operator Monitoring

Constant technological advancements in the area of Remotely Operated Vehicles (ROVs) and specifically Unmanned Aerial Vehicles (UAVs), facilitate their widespread use in a variety of safety-critical applications. Sometimes, the operator is required to control the vehicle in harsh and stressful situations. Stress and fatigue can be challenging factors that can compromise the outcome and safety of the mission. Therefore, the operator is prone to potentially issue involuntary movements to the controller of the UAV. Nevertheless, real-time monitoring of the operator and the UAV can prevent possible accidents. In this work, we present a feature-based Machine Learning (ML) approach for the classification of abnormal commands, via simultaneous monitoring of the human operator and the UAV, during a mission. The proposed classification targets onboard implementation on the UAV and operates in real-time. We provide experimental results using Jetson Xavier NX as an experimental platform that includes trade-offs across all the performance metrics, where we achieve 82% accuracy with minimal processing time (9ms) and energy consumption (5mJ).