Learning drone-control actions in surveillance videos

We address the problem of recognizing drone-control actions in surveillance videos. The goal is to identify malicious actions involving drone-controller interactions to understand ‘who is flying and controlling a drone to attack buildings’ from surveillance videos. The challenge is to learn the difference between actions with similar appearance as the human can perform control-actions with various objects (e.g., drone-controller vs. phone). To deal with this challenge, we propose a new model to learn human-object interaction actions with object context derived by human pose. We incorporates two networks for capturing both human action and its relevant object using Convolutional Neural Networks architecture. We validate our model on our new action dataset which includes drone-control actions, and show that our model outperforms other models.