Assessment of image generation and chemical reasoning in LLM chatbots, highlighting structural limitations.
This study evaluates the performance of freely accessible large language model (LLM) chatbots—ChatGPT, Copilot, Gemini, DeepSeek, and Perplexity—in a range of chemistry-related tasks, including molecular structure generation, Lewis structure construction, and chemical image interpretation. The results reveal a con-sistent limitation across all models: none were able to accurately generate chemical or Lewis structures, even after multiple attempts. Structural outputs often contained sig-nificant errors, rendering them unsuitable for educational use. However, despite these shortcomings in visual representation, the chatbots demonstrated strong conceptual understanding. All models correctly interpreted chemical structures presented in im-ages, identified functional groups, and accurately explained reaction mechanisms and molecular properties such as polarity and hybridization. Notably, they successfully recognized the reduction of allicin to a disulfide and identified conjugated systems as the source of color in natural dyes. These findings underscore the current strengths of LLMs in chemical reasoning and interpretation, while highlighting the need for further development in their structural visualization capabilities.
No takes yet. Share an insight, caveat, or question.
Endler Marcel Borges (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: