CollabSearch: A Study of User-LLM Collaboration in Task-Based Search

With the rapid proliferation of large language models (LLMs), users are increasingly turning to these systems to fulfill their everyday information needs. Unlike traditional search engines, which rely on structured query-response mechanisms, LLMs offer direct answer synthesis—fundamentally altering how users access and interact with information. However, the effectiveness of such direct responses and how users interact with such answers were not studied. In this work, we present CollabSearch, a user-LLM collaborative search system that enables users to interact with LLMs through adaptive role-playing prompts designed to guide and refine search outcomes. Users leverage the internal knowledge of LLMs while also benefiting from retrieval-augmented generation (RAG) to ensure relevance and currency. We evaluate our system through a task-based study involving 24 participants. The results demonstrate the effectiveness of our approach and offer nuanced insights into user–LLM interaction dynamics in search contexts.