What question did this study set out to answer?

The aim is to develop an approach for evaluating the functional correctness of AI products using statistical confidence.

January 25, 2026Open Access

Artifacts: Statistical Confidence in Functional Correctness: An Approach for AI Product Functional Correctness Evaluation

Key Points

The aim is to develop an approach for evaluating the functional correctness of AI products using statistical confidence.
Conducted interviews with participants to gather insights.
Utilized a Miro board for collaborative evaluation sessions.
Presented a runnable mock example of the proposed approach in a Jupyter notebook.
Collected and analyzed transcripts from multiple participant interviews to inform the evaluation method.
Created a mock example demonstrating the application of the proposed approach.
Provided supplementary materials to facilitate further understanding of the evaluation process.

Abstract

This repository contains the supplementary materials for the paper “Statistical Confidence in Functional Correctness: An Approach for AI Product Functional Correctness Evaluation”, submitted to CAIN 2026. The included. zip archive includes the interview transcripts, the Miro board used during the evaluation sessions, and a runnable mock example of the proposed approach. Filename Description Transcription Interview - P1. pdf Transcript of the P1 interview. Transcription Interview - P2. pdf Transcript of the P2 interview. Transcription Interview - P3. pdf Transcript of the P3 interview. Transcription Interview - P4. pdf Transcript of the P4 interview. SCFC Miro Board. pdf PDF export of the Miro board used during participant evaluations. SCFCApproachExample. ipynb A Jupyter notebook containing a mock example of the proposed approach, runnable on Google Colab (or locally, if all required libraries are installed).

Artifacts: Statistical Confidence in Functional Correctness: An Approach for AI Product Functional Correctness Evaluation

Key Points

Abstract

Cite This Study