What question did this study set out to answer?

The aim is to introduce a new framework that measures the robustness of large language models in challenging conditions.

February 2, 2026Open Access

Structural–Relational Cognitive Robustness: A Failure-Oriented Evaluation Framework for Large Language Models

Key Points

The aim is to introduce a new framework that measures the robustness of large language models in challenging conditions.
Developed the Structural–Relational Cognitive Robustness (SCRC) framework
Conducted experiments to evaluate models under epistemic stress
Provided complete evaluation code and logs for reproducibility
Aggregated metrics to report on model stability
Demonstrated that traditional accuracy measures may not capture model resilience under stress
Revealed variations in stability across different tested large language models
Facilitated reproducibility and verification of findings through provided resources

Abstract

This repository accompanies the manuscript submitted to the Journal of Machine Learning Research (JMLR): “Structural–Relational Cognitive Robustness: A Failure-Oriented Evaluation Framework for Large Language Models.” The project introduces SCRC (Structural–Relational Cognitive Robustness), a failure-oriented evaluation framework designed to measure relational stability of large language models under epistemic stress, rather than task accuracy. The repository contains: Complete evaluation code used in the experiments Full experimental logs for all tested models Aggregated metric tables reported in the manuscript Task definitions and prompt structures Instructions to fully reproduce all reported results No external datasets are required. All experiments are deterministic and fully specified. This archive is intended to ensure transparency, reproducibility, and independent verification of the reported findings. GitHub repository: https://github.com/hideyuki49/llm-cognitive-stability

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper