What question did this study set out to answer?

The central aim is to establish a framework for auditing fairness and mitigating bias in machine learning models applied in public organizations.

July 27, 2026Open Access

Fairness and transparency in ML: a methodological framework

NVNelson Salazar ValdebenitoAdolfo Ibáñez University GRGonzalo A. RuzAdolfo Ibáñez University RTReinel Tabares-SotoUniversity of Caldas

Key Points

The central aim is to establish a framework for auditing fairness and mitigating bias in machine learning models applied in public organizations.
Developed a methodological framework for fairness auditing and bias mitigation in supervised classification.
Conducted two experiments using data from Chile’s Public Criminal Defence Office to assess model performance.
Compared a base model with improved models utilizing reweighting, class weighting, threshold optimisation, and interpretability analysis.
In the drug trafficking experiment, false negative rate decreased from 40.54% to 31.63%, while false positive rate increased from 0.13% to 39.25%.
The framework demonstrated a clear trade-off between fairness and performance, illustrated through confusion matrices and statistical comparisons.

Abstract

As AI and ML applications become more common in public organisations, technical processes are needed to audit model behaviour, document limitations, and mitigate measurable disparities before deployment. This study presents a methodological framework for fairness auditing and bias mitigation in supervised classification, with emphasis on error disparities across groups rather than a complete solution to AI ethics. The framework is illustrated with two experiments based on data from Chile’s Public Criminal Defence Office (DPP), where the task is to predict whether a criminal case has a favourable or unfavourable outcome for the defendant. Each experiment compares a base model with an improved model that uses reweighting, class weighting, threshold optimisation, interpretability analysis, and documentation. Fairness is operationalised through group-level error metrics, especially false negative rate (FNR), false omission rate (FOR), and true positive rate (TPR), using Region as a geographic proxy attribute. The results show that mitigation can reduce selected disparities, but also reveal a clear fairness–performance trade-off. In the drug trafficking experiment, the manuscript-level confusion matrices show a reduction in FNR from 40.54 to 31.63%, with an increase in FPR from 0.13 to 39.25%. Statistical comparisons are computed from the confusion-matrix counts reported in this manuscript and are therefore reported as aggregate two-proportion comparisons rather than paired McNemar tests. The framework should therefore be read as a technical audit and mitigation procedure, not as evidence that the resulting models are suitable for operational legal decision-making.

Ask AI

Helpful

Bookmark

View Full Paper