• We propose a dynamic step-wise uncertainty estimation method for chemical problems. • We propose a collaborative framework for chemical problem solving. • Experiments across LLMs and datasets show ChemAU improves chemistry reasoning. Large language models (LLMs) have demonstrated remarkable reasoning capabilities and natural language understanding, leading to the widespread adoption across diverse applications. However, their effectiveness diminishes considerably when applied to chemistry-related problems, which involve specific terminology, chemical notation systems, and complex nomenclature conventions. These unique characteristics pose challenges for LLMs, which are primarily trained on general corpora with limited chemistry-specific data, resulting in inadequate chemical knowledge and hallucinations during reasoning. Existing methods remain insufficient to fully address these limitations. To bridge this gap, we propose ChemAU , a collaborative framework that integrates general and specialized LLMs for chemical reasoning. Our framework introduces a novel dynamic step-wise uncertainty estimation method tailored for the chemistry domain. This method precisely identifies chemical knowledge deficiencies in general LLM reasoning, after which the framework facilitates targeted knowledge supplementation via the chemistry-specific LLM. Experimental evaluations with widely-used LLMs across multiple chemical datasets demonstrate that ChemAU significantly enhances both reasoning accuracy and uncertainty estimation. Code is available at https://github.com/xinyi23/ChemAU .
Liu et al. (Thu,) studied this question.