Target testing and the PicHunter Bayesian multimedia retrieval system

We address how the effectiveness of a content-based, multimedia information retrieval system can be measured, and how such a system should best use response feedback in performing searches. We propose a simple, quantifiable measure of an image retrieval system's effectiveness, "target testing", in which effectiveness is measured as the average number of images that a user must examine in searching for a given random target. We describe an initial version of PicHunter, a retrieval system designed to test a novel approach to relevance-feedback. This approach is based on a Bayesian framework that incorporates an explicit model of the user's selection process. PicHunter is intentionally designed to have a minimal, "queryless" user interface, so that its performance reflects only the performance of the relevance feedback algorithm. The algorithm, however, can easily be incorporated into more traditional, query-based systems. Employing no explicit query, and only a small amount of image processing, PicHunter is able to locate randomly selected targets in a database of 4522 images after displaying an average of only 55 groups of 4 images. This is more than 10 times better than random chance.