A probabilistic framework for semantic indexing and retrieval in video

This paper proposes a novel probabilistic framework for semantic indexing and retrieval in digital video. The components of the framework are multijects and multinets. Multijects are probabilistic multimedia objects (Naphade et al., 1998) representing semantic features or concepts. A multinet is a probabilistic network of multijects which accounts for the interaction between concepts. The main contribution of this paper is a Bayesian multinet which enhances the detection probability of individual multijects, provides a unified framework for integrating multiple modalities and supports inference of unobservable concepts based on their relation with observable concepts. We develop multijects for detecting sites (locations) in video and integrate the multijects using a multinet in the form of a Bayesian network. Experiments reveal significant performance improvement using the multinet.