Performance of ChatGPT-5.5, Gemini 3, and copilot 2026 in answering chemistry course questions: a cross-sectional study

Chemistry, environmental health, and occupational health have historically developed in close collaboration. This study aimed to evaluate the performance of LLMs, in responding to a chemistry course questionnaire used in environmental and occupational health department for educational purposes. This was a cross-sectional study conducted in 2026. The study used a 50-item multiple-choice questionnaire, primarily in Persian with limited English terms and formulas, which was administered to LLMs (ChatGPT-5.5, Gemini 3, and Copilot 2026). Performance was evaluated using confusion matrix analysis. Cochran`s Q test, McNemar`s test for pair-wise comparisons, and Point-biserial correlation were also conducted to further evaluate the study data. The LLMs presented mean sensitivity, specificity, and accuracy of 0.81, 0.29, and 0.68, respectively. Gemini 3 presented a high level of performance, achieving the highest sensitivity (0.85), specificity (0.56), and accuracy (0.80). ChatGPT-5.5 had the lowest sensitivity (0.76), while Copilot showed the lowest specificity (0.07) and an accuracy of 0.62. No significant correlations were found between performance and questionnaire characteristics; however, all models performed poorly on negatively framed items requiring identification of incorrect options. The study presented some preliminary insights on the performance of LLMs in chemistry-related questions. Further research is warranted.