Machine learning (ML) models are increasingly deployed in various safety-critical applications, e.g., as assistants in medical diagnostics for early melanoma detection. Despite their impressive predictive capabilities, these models often exploit spurious correlations present in the training data, as they are trained to detect any pattern in the data to solve the underlying task. For instance, ML models for medical imaging may base predictions on the existence of data artifacts, such as hospital tags, rather than clinically valid features, which can result in potentially dangerous mispredictions in high-stakes scenarios. The detection and correction of such unintended behavior pose significant challenges, as existing methods typically require extensive annotations or significant human supervision, which are often impractical to obtain in real-world applications. This thesis addresses these challenges by introducing the Reveal2Revise framework, an iterative and interpretability-driven model debugging life cycle designed to minimize the reliance on human intervention. The framework consists of four key steps: bias revealing, bias modeling, model revision, and evaluation. We further identify and address shortcomings in existing methods utilized for the first three steps. Specifically, we review and compare interpretability-driven approaches for detecting spurious model behavior and introduce semi-automated methods for bias annotation to reduce manual effort. Furthermore, we develop robust pattern-based concept activation vectors for precise bias modeling and propose a novel bias mitigation approach that penalizes spurious model behavior in the latent space. Our contributions enhance the reliability and trustworthiness of ML models, ultimately facilitating their responsible deployment in safety-critical applications.
Frederik Pahde (Thu,) studied this question.