Semi-automatic video annotation based on active learning with multiple complementary predictors

In this paper, we will propose a novel semi-automatic annotation scheme for video semantic classification. It is well known that the large gap between high-level semantics and low-level features is difficult to be bridged by full-automatic content analysis mechanisms. To narrow down this gap, relevance feedback has been introduced in a number of literatures, especially in those works addressing the problem of image retrieval. And at the same time, active learning is also suggested to accelerate the converging speed of the learning process by labeling the most informative samples. Generally an active learning scheme includes a sample selection engine and a learning engine. In this paper, we will discuss the limitations of existing active learning algorithms and propose a novel active learning scheme based on multiple complementary predictors and incremental model adaptation, which improves the efficiencies of both of the primary components of active learning. Firstly, an efficient sample selection scheme is proposed, in which multiple predictors are applied to find most informative samples. Then an incremental model adaptation technique, maximum likelihood linear regression (MLLR), is used to update the classifiers which tackle the issue of unbalance between the original training set and the newly labeled data. It is proved that the samples selected by the proposed scheme are more representative than general active learning scheme, as well as the incremental model adaptation scheme is effective especially when the newly added data size is small.