Fast semantic image retrieval based on random forest

This paper introduces random forest as a computational and data structure paradigm for fusing low-level visual features and high-level semantic concepts for image retrieval. We use visual features to split the tree nodes and use the image labels to supervise the splitting to make images located at the same tree node share similar semantic concepts as well as visual similarities. We exploit such a random forest and define the semantic neighbor set (SNS) of a given image as the union of all images in the leaf nodes that this image falls onto. From SNS we further define the semantic similarity measure (SSM) between two images as the number of trees in which they share the same leaf nodes within a SNS. With SNS and SSM, example-based image retrieval becomes that of first finding the SNS of the querying image and then ranking the images according to the SSMs between the querying image and images in its SNS. We also show that the new technique can be adapted for keyword-based semantic image retrieval. The inherent efficient tree data structure leads to fast solutions. We will present experimental results to show the effectiveness of this new semantic image retrieval technique.