Improving automatic query expansion

Most casual users of IR systems type short queries. Recent research has shown that adding new words to these queries via blind feedback, without any input from the user, improves the performance of such queries. We investigate ways to improve this query expansion process by refining the set of documents used in feedback. We start by using manually formulated Boolean filters along with proximity constraints. Our approach is similar to the one proposed in 10. Next, we investigate a completely automatic method that makes use of term cooccurrence information to estimate word correlation. Results show that refining the set of documents used in query expansion yields substantial improvements in retrieval effectiveness, both in terms of average precision and precision at top twenty documents. Such refinement often prevents the query drift caused by blind expansion. More importantly, the fully automatic approach developed in this study performs competitively with the best manual approach and...