Efficient Filtering and Clustering Methods for Temporal Video Segmentation and Visual Summarization

Automatic temporal segmentation and visual summary generation methods that require minimal user interaction are key requirements in video information management systems. Clustering presents an ideal method for achieving these goals, as it allows direct integration of multiple information sources. This paper proposes a clustering-based framework to achieve these tasks automatically and with a minimum of user-defined parameters. The use of multiple frame difference features and short-time techniques are presented for efficient detection of cut-type shot boundaries. Generic temporal filtering methods are used to process the signals used in shot boundary detection, resulting in better suppression of false alarms. Clustering is also extended to the key frame extraction problem: Color-based shot representations are provided by average and intersection histograms, which are then used in a clustering scheme to identify reference key frames within each slot. The technique achieves good compaction with a minimum number of visually nonredundant key frames.