Qualifying Software Changes Without Overstating Evidence: A controlled diagnostic study of evaluator failure and controller composition

Software-change evidence can be genuine yet insufficient for the conclusion drawn from it. We examine this problem in a controlled diagnostic study of an author-directed evaluator-development effort assisted by artificial intelligence (AI) and a bounded comma-separated values (CSV) repair instance, BX01. Retrospectively selected counterexamples concern report origin, execution phase, program semantics, execution profile, event coverage and input domain. We distinguish incorrect local observations from valid observations used outside their scope. Two separately implemented controllers assess the same thirteen known transaction requests using shared admission and native-observation components. A fixed domain contains 584 inputs; complete goal verification requires all of them, whereas rejected, incomplete or faulty cases may supply fewer observations. Each controller returns two PASS, eight FAIL and three BLOCKED verdicts, with identical values in ten recorded assessment fields. Four changes pass all functional inputs but violate other transaction requirements. We report excluded implementations that meet the checked targets and descriptive timing; one timing difference reverses sign on replay. The study provides traceable diagnostic contrasts, counterexample-linked review questions and a bounded controller-composition check. It establishes neither a method-superiority result nor general correctness of the observers. Failure prevalence, independent-observer effectiveness, live-agent behavior and cross-project generalization remain untested.