Face detection in 360-degree video using hybrid augmentation method and YOLOv8 detector

Face detection in 360-degree videos is crucial for various applications, including navigation, interaction, and analysis. However, accurately detecting faces in these videos presents unique challenges due to visual distortions that can make faces appear disproportionate and difficult to recognize. While deep learning algorithms like You Only Look Once (YOLO) have shown promise in conventional video analysis, their direct application to 360-degree videos often yields suboptimal performance. This study introduces a novel image augmentation strategy that combines distortion-aware transformations to address the specific challenges of face detection in 360-degree video content. The research consists of four main stages: preprocessing, dataset augmentation, model training and hyperparameter tuning, and object detection. The proposed method was evaluated against YOLO’s standard augmentation and a previous equirectangular-based augmentation method using three videos each in Cubemap Projection (CMP) and Equirectangular Projection (ERP) formats. The results demonstrate strong performance, achieving the highest average confidence score in 83% of the cases, indicating its effectiveness in handling the challenges of 360-degree environments. While the model produces fewer detections, those identified have higher confidence scores. Qualitative analysis further supports its superiority over standard YOLO and previous methods, although it struggles with extreme rotations. This research highlights the importance of developing image augmentation strategies to address the unique challenges of face detection in 360-degree videos.