Chatbot accuracy in bioinformatics knowledge and coronavirus sequence images

Introduction: Most chatbot studies have focused on textual-based questions. The performance of chatbots on bioinformatics and their capabilities in interpreting sequence image-based results remain largely unexplored. Materials and methods: We compared the performance and runtime of five chatbots (ChatGPT, Perplexity, Claude, Copilot, and Gemini), including both free and subscribed versions of ChatGPT, Perplexity, and Claude, in answering 1120 questions on theory-based bioinformatics knowledge and image-based interpretation of coronavirus phylogenetic trees and protein structure figures. Results: The median score on theory-based question subsets (100%) was significantly higher than image-based subsets (95%). For image-based questions, the ChatGPT-5 free (97.5%) and subscribed (97.5%) versions scored significantly higher than other chatbots. The score on multiple sequence alignment (MSA) tools (80%) was significantly lower than phylogeny (100%) and structure tools (95%). Chatbots took much longer on image-based questions, mainly due to the exceptionally long computational time for ChatGPT-5.1 free. There was no significant difference between the free and subscribed versions. Conclusions: Chatbots were competent in answering theory-based questions, showing high versatility. However, they showed relatively poorer performance on MSA and specific categories of image-based questions. These findings highlight both the strengths and limitations of chatbots in bioinformatics and image interpretation.