Visualization of Subjects and Their Motion in Video Using Principal Component Analysis

In recent years, videos have become an integral part of daily life and there is an enormous volume of video content. As a result, finding specific videos or scenes within this vast amount of content has become a difficult task. In order to enable quick scene recognition by visualizing both video content and relationships between frames, the authors introduce a method for visualizing the subjects and their motion by arranging video frames in a spatial layout based on their correlations. This method calculates frame correlation matrix by applying Principal Component Analysis (PCA) and arranges the video frames in a spatial layout based on the principal component scores. From the visualization results by scatter plots and 3D space, they found that the scatter plot of minimal frames and 3D space arrangement of video frames were more effective than traditional video playback for scene recognition and workload reduction.