Evaluating Safety of Large Language Models with Cognitive Diagnosis

This study introduces a novel approach to evaluating the safety of large language models by applying cognitive diagnosis, a tool traditionally used in educational platforms. The key innovation lies in moving beyond the restrictive “one exercise–one attribute” paradigm employed in nearly all contemporary safety benchmarks. We propose utilizing cognitive diagnosis to validate large language model safety, create an automated method for attribute extraction, and demonstrate its superiority over the original annotation of the XSTest safety benchmark in terms of predictive capability. We address a critical issue: prompts typically contain multiple requirements for safety knowledge, rather than a single one. For instance, a prompt concerning historical violence may simultaneously involve ethical reasoning, harmful content detection, cultural sensitivity, and factual accuracy. Our methodology reduces large language model safety evaluation to a cognitive diagnosis task by treating models as “students,” prompts as “exercises,” and binary scores as “responses.” This enables the identification of latent levels of large language model safety knowledge within a multiattribute framework. The proposed automatic attribute discovery process represents progress toward creating interpretable safety taxonomies. The resulting 11 attributes provide a more granular characterization than the original eight categories from the XSTest benchmark, demonstrating improvements in predictive metrics. These findings indicate the superiority of multiattribute question annotation over single-attribute annotation in safety evaluation.