Form and function: The impact of query term and operator usage on Web search results

Abstract Conventional wisdom holds that queries to information retrieval systems will yield more relevant results if they contain multiple topic‐related terms and use Boolean and phrase operators to enhance interpretation. Although studies have shown that the users of Web‐based search engines typically enter short, term‐based queries and rarely use search operators, little information exists concerning the effects of term and operator usage on the relevancy of search results. In this study, search engine users formulated queries on eight search topics. Each query was submitted to the user‐specified search engine, and relevancy ratings for the retrieved pages were assigned. Expert‐formulated queries were also submitted and provided a basis for comparing relevancy ratings across search engines. Data analysis based on our research model of the term and operator factors affecting relevancy was then conducted. The results show that the difference in the number of terms between expert and nonexpert searches, the percentage of matching terms between those searches, and the erroneous use of nonsupported operators in nonexpert searches explain most of the variation in the relevancy of search results. These findings highlight the need for designing search engine interfaces that provide greater support in the areas of term selection and operator usage.