Analysis reveals performance gaps in molecular property prediction between language and machine learning models, suggesting implications for chemistry applications.
Large language models (LLMs) demonstrate strong performance in natural language tasks, but their capacity for genuine in‐context learning (ICL) in scientific regression remains unclear. We systematically assessed seven LLMs on molecular property prediction using a controlled framework of 56 transformed tasks that isolate shortcut learning and are designed to induce functional out‐of‐distribution (OOD) behavior. LLMs performed nearly perfectly on raw molecular weight prediction via shortcut cues but deteriorated under nonlinear transformations, whereas machine learning (ML) baselines showed greater robustness, yielding a performance crossover. Meta‐analysis revealed that distributional descriptors and structure–activity landscape indices (SALI) predict task favorability, providing a framework for selecting between LLM‐ and ML‐based approaches in chemistry.
No takes yet. Share an insight, caveat, or question.
Joe et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: