Answers are grounded in this paper and cite their sources.
ChemTorch unifies the benchmarking of deep learning models for chemical reactions, revealing flaws in traditional evaluation methods.
The accurate prediction of chemical reaction properties, such as yields and activation energies, is fundamental to chemical synthesis and industrial process optimization. In recent years, deep learning has emerged as a powerful tool for this task, achieving accuracies comparable to traditional quantum mechanical methods at a fraction of the computational cost. Models tackling different tasks, from reaction outcome prediction to retrosynthesis, have been developed, leveraging different representations like molecular graphs or text-based notations. However, this progress has led to a fragmented ecosystem of specialized, incompatible codebases. The lack of a unified framework and standardized evaluation protocols makes it difficult to compare methods fairly, limits the development of new approaches, and hinders the assessment of the true real-world performance of models. Here, we introduce ChemTorch, a modular open-source framework designed to unify and streamline the development and benchmarking of deep learning models for chemical reactions. Using ChemTorch, we conduct a systematic study on reaction barrier height prediction and demonstrate how common but unrealistic benchmarking practices, such as providing ground-truth atom-to-atom mappings or 3D geometries at test time, lead to overly optimistic performance estimates. By re-evaluating models in a setting that withholds such privileged information, we reveal significant performance drops that better reflect the challenges of real-world deployment. We therefore propose a new set of benchmarking guidelines that promote end-to-end evaluation, where models must operate on readily available 2D chemical structures without relying on computationally expensive data. ChemTorch provides the community with a powerful tool to accelerate research and promote reproducibility, and by adopting these more rigorous evaluation standards, the field can shift its focus towards developing models that are not only accurate in benchmarks but are genuinely applicable in practice.
No takes yet. Share an insight, caveat, or question.
Landsheere et al. (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: