Behavior detection based on infrared video: review and prospect

Human behavior detection holds diverse research value across various scenarios and tasks, with promising application in intelligent security, virtual-reality, and human-computer interaction. Human behavior detection is a data analysis field involving multi-modal perception, which covers different data modes such as visible light, skeleton, depth map, infrared, point cloud and so on. In the field of computer vision, human behavior detection can be divided into two categories according to the type of data processed: visible light video and infrared video. Due to the minimal impact of lighting on infrared imaging technology, it can not only achieve continuous imaging workday and night but also provide better imaging images during nighttime when criminal activities are likely to occur. Therefore, a comprehensive and systematic summary of the field is of utmost importance. Firstly, according to the difference of feature learning method, it is divided into manual feature extraction method based on traditional machine learning and deep feature extraction method based on deep learning; for deep feature extraction methods, methods based on convolutional neural networks and their improved versions, human skeletal data networks, attention subject of behavior networks, and optimized infrared image networks are analyzed. Secondly, mainstream infrared human behavior datasets at home and abroad are reviewed. Finally, the limitations of existing research methods are summarized from aspects such as available datasets, data fusion, performance, and future research directions and optimization suggestions are provided. The study results show that the four types of infrared human behavior detection methods each have their advantages and disadvantages. Future research should focus on the combination of large-scale unsupervised learning and transfer learning, enhance the model's adaptability and generalization ability in new scenarios; explore multi-modal data fusion methods to improve the accuracy and robustness of behavior detection; design lightweight models and optimization algorithms to enhance real-time performance and computational efficiency.