Beyond one-time relabeling: A sample inspection framework for correcting noisy labels in both classification and regression tasks

Label noise remains a fundamental challenge in supervised learning, often degrading model performance and data reliability. While many existing approaches assume that a single relabeling action by a human annotator suffices to correct errors, this assumption overlooks the inherent fallibility of human judgment. In this paper, we propose a general-purpose framework that models relabeling as an iterative and imperfect process, drawing inspiration from industrial sample inspection techniques. Our method integrates uncertainty sampling with targeted inspection to identify and refine mislabeled data through repeated annotation and aggregation. Crucially, this framework extends label correction beyond categorical classification to encompass real-valued regression tasks. For categorical data, majority voting is used to resolve annotation conflicts, while for continuous labels, we introduce an averaging-based correction strategy that incrementally approximates true values over successive relabeling rounds. This design enables systematic refinement of both discrete and continuous labels under noisy supervision. We validate our approach through a two-stage evaluation. First, on clean benchmark datasets with synthetic noise, we demonstrate significant improvements in relabeling efficiency under various uncertainty metrics and model capacities. Second, on a real-world image dataset with inherently noisy labels, our method continues to outperform baseline strategies, especially when paired with strong learners like XGBoost. Our findings show that sample inspection is a scalable and cost-effective mechanism for robust label correction across both classification and regression domains, even under high error rates or limited computational resources.