Large Language Models (LLMs) as Search Engine Alternatives for Improved Access and Quality of Health-related Information: A Qualitative Investigation into Experiences of Patients with Severe Congenital Scoliosis and Families (Preprint)

BACKGROUND In the digital age, access to health information has become a fundamental resource for patients managing chronic conditions such as severe congenital scoliosis. Traditional search engines often fall short in providing reliable, tailored information. Large Language Models (LLMs), with their advanced capabilities in natural language processing, offer a promising alternative by delivering personalized, context-sensitive information. OBJECTIVE To investigate the experiences and perception of patients with severe congenital scoliosis and families in using two selected Chinese LLMs with internet-browsing capability. METHODS In this qualitative phenomenological study, we introduced two pre-selected LLMs to 121 patients and family members during routine clinical care and interviewed 12 participants enrolled by purposive sampling. Face-to-face, semi-structured interviewing was conducted to solicit their experiences and perceptions. Data were analyzed using Colaizzi's method. RESULTS Four main themes were identified, including "replacement of conventional search engines," where participants reported a unanimous shift among participants from traditional search engines to LLMs driven by the LLMs’ superior ability to handle natural language queries and provide contextually relevant information; "improved access and quality of information" characterized by the 24/7 availability of LLMs and their capability to significantly improve the quality of information retrieved; "importance of prompting skills," where the effectiveness of LLMs was highly dependent on users' ability to formulate questions accurately, a skill that required learning and adaptation and influence user satisfaction and engagement; and "challenges with specificity and accuracy in LLM responses," which pointed to the limitations in the LLMs' performance, particularly when dealing with highly specific queries or when asked to recommend specific treatments, where errors were more likely and adherence to ethical guidelines restricted the provision of certain types of advice. CONCLUSIONS Patients with severe scoliosis and families using LLMs for health-related information seeking tend to develop a clear preference for LLMs over conventional search engine, due to LLMs’ ability to provide accessible, personalized, and context-aware information on-demand. They demonstrate a significant behavioral shift towards more interactive and responsive forms of information retrieval. However, challenges such as inaccuracies in responses to specific queries and ethical limitations in providing treatment recommendations suggest the need for ongoing technological enhancements.