From Click to Chat: A Comparative, Bilingual Study of Autism Related Information on Websites and General Purpose AI Chatbots (Preprint)

BACKGROUND Parents increasingly consult the internet —both websites and, more recently, AI chatbots—for information on autism spectrum disorder (ASD). Yet the comparative quality of these two source types, especially across languages, remains under-explored. OBJECTIVE We aimed to assess the completeness and accuracy of ASD information delivered by websites and five popular AI chatbots, and determine whether performance differs between English and Romanian content. METHODS In a cross-sectional design, we evaluated 25 English-language and 25 Romanian-language websites and the responses of ChatGPT, Gemini, Claude, Copilot, and DeepSeek. Content was benchmarked against a 24-item checklist, yielding completeness and accuracy scores. Chatbots were tested in two scenarios: a single broad query and 24 item-specific queries. RESULTS Websites achieved higher completeness in English than in Romanian (6.9 vs 5.1; p=0.007) and marginally higher accuracy (6.9 vs 6.1; p=0.045). In Scenario A, chatbots’ completeness (English 5.3 vs Romanian 6.2; p= 0.151), and accuracy (English 6.0 vs Romanian 5.6; p= 0.322) did not show significant differences by language. In the single-query scenario, websites showed higher accuracy than chatbots in both English (6.9 vs 6.0, p=0.19) and Romanian (6.1 vs 5.6, p=0.532), with neither difference reaching statistical significance. Conversely, item-specific questioning favored chatbots, which outperformed websites in English (8.3 vs 6.9, p=0.053) and—significantly—in Romanian (8.0 vs 6.1, p=0.007). Accuracy rose significantly from the single-query to the item-specific scenario (English: 6.0 vs 8.3, p=0.002; Romanian: 5.6 vs 8.0, p=0.001). Between-chatbot variation was statistically significant (One-way repeated measures ANOVA p=0.005), with Gemini and DeepSeek showing superior scores (p=0.016 and 0.023 respectively). CONCLUSIONS The quality of online ASD information varies by source type and language. English-language websites were more complete than Romanian ones. Targeted questioning of chatbots yielded significantly more accurate answers in both languages.