Time-constrained clustering for segmentation of video into story units

Many video programs have story structures that can be recognized through the clustering of video contents based on low-level visual primitives, and the analysis of high level structures imposed by temporal arrangement of composing elements. In this paper time-constrained clustering of video shots is proposed to collapse visually similar and temporally local shots into a compact structure. We show that the proposed clustering formulations, when incorporated into the scene transition graph framework, allows the automatic segmentation of scenes and story units that cannot be achieved by existing shot boundary detection schemes. The proposed method is able to decompose video into meaningful hierarchies and provide compact representations that reflect the flow of story, thus offering efficient browsing and organization of video.