Large Language Model for Automated Scientific Hypothesis and Evidence Analysis

The rapid increase in the amount of scientific literature makes it increasingly difficult to find and identify core scientific hypotheses, experimental designs, and the relationship between those hypotheses and designs in order to accelerate knowledge discovery. Manual scans through scientific articles to identify content around scientific hypotheses are inefficient, and although Large Language Models (LLMs) demonstrate potential in processing literature, they have well-known challenges (particularly in specialized scientific domains) associated with precision (i.e. hallucination) and structured reasoning. To address this, we introduce the Prompt-Enhanced LLM for Scientific Hypothesis Analysis (PEL-SHA) framework, which uses elaborate and meaningful multi-stage prompt engineering approaches to enable LLMs to automatically find, classify, and reason around scientific hypotheses, supporting evidence and methods through paper abstracts. Our framework consists of a sequential pipeline using Hypotheses Identification, Evidence and Method Classification, and Potential Research Direction Reasoning prompts. To rigorously test PEL-SHA, we introduce SciHypo-500, a new benchmark dataset containing 500 expert-annotated scientific abstracts. We conduct extensive experiments against the best performing LLMs to show that PEL-SHA is consistently superior against all evaluation tasks.