What question did this study set out to answer?

To evaluate the efficacy and detection limits of an operator self-auditing software defects using dedicated tooling versus independent peer review over a one-year timeframe.

August 19, 2026Open Access

Keeping Score: what a year of recording our own defects taught us about evidence for automated decisions

View Full Paper

Ask AI

Bookmark

Authors

IGIshaan Ghosh

Discussion

Member takes

Overview

Longitudinal audit reveals that external reviewers detected 55.8% to 56.8% of software defects during self-checking, indicating significant limitations in relying solely on self-attestation.

Key Points

To evaluate the efficacy and detection limits of an operator self-auditing software defects using dedicated tooling versus independent peer review over a one-year timeframe.
Tracked and recorded 95 numbered defect items generated by a single operator over a one-year evaluation period.
Classified defects using an empirical taxonomy consisting of ten distinct defect categories.
Quantified the proportion of defects independently detected by external reviewers using a three-way classification model.
The primary operator identified fewer than 50% of their own operational defects without third-party assistance.
External contributors identified between 55.8% and 56.8% of the 95 total logged defects, established as a baseline floor.

Cite This Study

Ishaan Ghosh (2026) studied this question.

synapsesocial.com/papers/6a8563d703308d306e2d7389 https://doi.org/10.5281/zenodo.21977571

View Full Paper

Ask AI

Bookmark

August 19, 2026Open Access

Keeping Score: what a year of recording our own defects taught us about evidence for automated decisions

View Full Paper

Ask AI

Bookmark

Authors

IGIshaan Ghosh

Discussion

Member takes

Overview

Longitudinal audit reveals that external reviewers detected 55.8% to 56.8% of software defects during self-checking, indicating significant limitations in relying solely on self-attestation.

Key Points

To evaluate the efficacy and detection limits of an operator self-auditing software defects using dedicated tooling versus independent peer review over a one-year timeframe.
Tracked and recorded 95 numbered defect items generated by a single operator over a one-year evaluation period.
Classified defects using an empirical taxonomy consisting of ten distinct defect categories.
Quantified the proportion of defects independently detected by external reviewers using a three-way classification model.
The primary operator identified fewer than 50% of their own operational defects without third-party assistance.
External contributors identified between 55.8% and 56.8% of the 95 total logged defects, established as a baseline floor.

Cite This Study

Ishaan Ghosh (2026) studied this question.

synapsesocial.com/papers/6a8563d703308d306e2d7389 https://doi.org/10.5281/zenodo.21977571

View Full Paper

Ask AI

Bookmark