Frontier AI Safety Confidence Evaluate

Frontier AI systems bring societal benefits while also posing new risks, prompting AI developers and regulators to explore safety assurance methods. Safety cases—structured arguments supporting top-level claims about a system’s safety properties—are a viable option among these methods. However, such top-level binary claims (e.g., "Deploying the AI system involves no unacceptable risk") are difficult to state clearly in practice, making it necessary to define their confidence levels. This study adopts the Assurance 2.0 safety assurance methodology and applies it to the frontier AI inability argument addressing cyber misuse harms. It finds that numerical quantification of confidence is challenging, but the process of generating such assessments can improve safety cases; meanwhile, it proposes a Delphi method implemented purely by LLMs to enhance the reproducibility and transparency of probabilistic assessments of confidence in argument leaf nodes. Additionally, it provides AI developers with a method to prioritize argument defeaters (doubts or flaws in safety cases) to improve investigation efficiency and offers recommendations for communicating confidence information to decision-makers.