This analysis of AI-enhanced search applications reveals challenges in evaluating drug information, suggesting implications for media literacy and application design.
AI-enhanced search applications are evolving rapidly, promising to answer complex queries with simple answers, but this creates new challenges on how users evaluate the information presented to them. This study investigates how four AI-enhanced search applications (ChatGPT, Gemini, Copilot and You.com) respond to questions about erectile dysfunction drugs in Sweden, which risks creating digital grey zone markets in the search results. By analysing the conversations and application interfaces, the study finds two different methods for integrating search and AI: search-first and AI-first. Search-first methods often reflect or enhance the issues from search engine results, while AI-first methods, although better at handling nuance, would often refuse to respond in a helpful manner due to moderation. Surprisingly, more advanced language models did not always deliver better results; instead, application design choices appeared to influence how they handle real-world and sensitive scenarios. The findings have implications for media literacy, application auditing practices and the development of AI search applications.
No takes yet. Share an insight, caveat, or question.
Söderström et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: