Answers are grounded in this paper and cite their sources.
This evaluation assesses LLMs' performance in classifying scientific uncertainty in climate change statements, highlighting their limitations.
Large Language Models (LLMs) have shown remarkable performance across various natural language processing tasks, but their ability to interpret and classify scientific uncertainty remains underexplored. This study evaluates the performance of four recent LLMs-GPT-4o-mini, GPT-4o, GPT-4-turbo, and GPT-4.5-preview-on the ClimateX dataset, which consists of climate science statements labeled with expert-assessed confidence levels. Using both zeroshot and few-shot prompting strategies, we assess each model's ability to classify statements into one of four IPCC-defined confidence categories: low, medium, high, and very high. Our findings indicate that while newer models like GPT-4.5-preview outperform earlier versions, overall classification accuracy remains modest (with a maximum of 51.33%). These results highlight the challenges LLMs face in interpreting nuanced scientific language and emphasize the need for further development and domain-specific adaptation to enhance their utility in climate communication and scientific analysis.
No takes yet. Share an insight, caveat, or question.
Pan et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: