Observational analysis reveals significant improvements in saliency prediction using audio-visual cues in 360-degree videos, suggesting integration of spatial audio is key.
Key Points
SalViT360 and SalViT360-AV significantly enhance saliency prediction in 360-degree videos, improving viewer attention understanding.
The YT360-EyeTracking dataset of 81 omnidirectional videos was curated to explore saliency prediction in unique audio-visual conditions.
Applying spherical geometry-aware attention layers helps effectively capture the complexities in predicting viewer focus in 360-degree environments.
Integrating spatial audio cues in the model architecture is crucial for optimal saliency prediction in omnidirectional videos.