The Socratic Stress Test (SST): A Structured Framework for Auditing Epistemic Reliability

Large Language Models (LLMs) can produce coherent answers across diverse domains but remain prone to inconsistencies, unsupported claims, and epistemic overconfidence. Evaluating their reasoning processes requires structured methods that distinguish between factual knowledge, speculative inference, and uncertainty. The Socratic Stress Test (SST) is a conceptual framework inspired by the Socratic method and designed to probe the reliability and transparency of LLM outputs. Through systematic questioning, the SST extracts implicit assumptions, links claims to supporting evidence, tests internal coherence, and forces abstention when information is insufficient. A central feature is the explicit classification of statements as FACT, HYP (hypothesis), or UNK (unknown), enabling reproducibility and clearer epistemic boundaries. We introduce the SST protocol, illustrate its application on an ambiguous claim, and discuss its potential for evaluating both artificial and human reasoning. The approach promotes transparency, fosters epistemic humility, and supports future research on cognitive assessment in hybrid human–AI systems. While formal empirical validation is ongoing, this paper offers a complete conceptual framework and an operational protocol, setting the stage for future research on transparent AI assessment.