Collecting relevance feedback on titles and photographs in weblog posts

We investigate new interfaces that allow users to specify topics of interest in streams of weblog stories by providing relevance feedback to a search algorithm. Noting that weblog stories often contain photographs taken by the blogger during the course of the narrated events, we investigate whether these photographs can serve as a proxy for the whole post when users are making judgments as to the post's relevance. We developed a new story annotation interface for collecting relevance feedback with three variations: users are presented either with the full post as it appears in a weblog, an embedded photograph, or only the title of the post. We describe a user evaluation that compares annotation time, quality, and subjective user experience across each of these three conditions. The results show that relevance judgments based on embedded photographs or titles are far less accurate than when reading the whole weblog post, but the time required to acquire an accurate model of the user's topic interest is greatly reduced.