Answers are grounded in this paper and cite their sources.
The trustworthiness of reasoning conclusions is a key criterion for determining whether scientific agents can progress from experimental demonstrations to real-world research applications, and it is essential for preventing conclusions with latent defects from entering experimental design and critical decision-making. Verification restricted to positive evidence coverage of an explicit reasoning chain can leave hidden premises and competing hypotheses untested and can be vulnerable to criterion-aware generation. Drawing on the falsificationist position in the philosophy of science, this paper redefines trustworthiness as the ability of a reasoning conclusion to survive continually evolving falsification attacks and proposes FalsifyAgent, a falsification-based verification framework that integrates backward abductive reconstruction with adversarial co-evolution. We develop a trustworthiness measurement system centered on the Adversarially Robust Credibility Score (ARS), which formalizes credibility as the area under the survival curve of a conclusion subjected to progressively stronger falsification attacks, replacing the conventional binary pass/fail judgment. Under uniform positive conditional bounds on attack coverage and correct final adjudication, we derive a decay bound for the expected ARS of a fixed false proposition. High finite-budget ARS indicates stronger resistance to the specified tests. We introduce a backward abductive reconstruction mechanism that starts from a conclusion and constructs an evidence-demand tree containing must-exist and must-not-exist nodes. This mechanism exposes hidden premises and evidential gaps that forward coverage checks cannot reach as dangling requirements, and combines competing-hypothesis generation with explanatory-power evaluation to actively identify evidential underdetermination and perform graded conclusion downgrading. We design a dynamic verification mechanism based on attacker–defender co-evolution and a library of composable attack operators. A conclusion-information constraint in the bidirectional fitness function prevents vacuous defense, while an evidence-bearing attack requirement prevents unsupported challenges. We also propose a weakness-guided coupling strategy that directs attacks toward high-value targets, including gap nodes revealed by the demand tree and competing hypotheses with small explanatory-score margins. Attack outcomes, in turn, trigger incremental reconstruction of the demand tree, forming a closed verification loop of weakness localization, targeted attack, and repair-driven update. Experiments on the FalsifySci benchmark, which contains 400 propositions spanning biomedicine, chemistry, physics, and computer science, reveal substantial performance differences among verification paradigms. With GPT-4o as the backbone, FalsifyAgent achieves an overall ARS-H of 72.7%, outperforming the shared-corpus retrieval fact-checker by 17.9 percentage points and the four-round debate baseline by 10.6 percentage points. On the hidden-premise subset, it attains a hidden-premise discovery rate of 71.3% with a false-positive rate of only 9.4%, while achieving a downgrade-calibration score of 81.0% on the main set. On the trap-proposition subset, the system penetrates the superficial self-consistency created by retracted literature and outdated data, reaching a false-claim rejection rate of 79.2% and reducing the false-acceptance rate by 21.0 percentage points relative to the shared-corpus retrieval fact-checker. Ablation experiments further show that coupling backward reconstruction and evolutionary adversarial testing through weakness guidance produces a superadditive gain of 1.5 percentage points, confirming the functional orthogonality and synergistic amplification of the framework components. FalsifyAgent provides a promising framework for auditing scientific-agent conclusions, supported by conditional theoretical analysis and evaluation on FalsifySci, with potential applications in prepublication review assistance, decision support, and benchmark-contamination screening. Code and implementation details are available at https://huggingface.co/wmsr22/FalsifySci/tree/main .
No takes yet. Share an insight, caveat, or question.
Xiao et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: