Brain-inspired warm-up training with random noise for uncertainty calibration

Uncertainty calibration, the alignment of predictive confidence with accuracy, is essential for the reliable deployment of machine learning systems in real-world applications. However, current models often fail to achieve this goal, generating responses that are overconfident, inaccurate or even fabricated. Here we show that the widely adopted initialization method in deep learning—long regarded as standard practice—is, in fact, a primary source of overconfidence. To address this problem, we introduce a neurodevelopment-inspired warm-up strategy that inherently resolves uncertainty-related issues without requiring pre- or post-processing. In our approach, networks are first briefly trained on random noise and random labels before being exposed to real data. This warm-up phase yields optimal calibration, ensuring that confidence remains well aligned with accuracy throughout subsequent training. Moreover, the resulting networks demonstrate high proficiency in the identification of ‘unknown’ inputs, providing a robust solution for uncertainty calibration in both in-distribution and out-of-distribution contexts. Cheon and Paik show that overconfidence in deep neural networks arises from standard initialization practices, and that brief warm-up training with random noise improves uncertainty calibration and meta-cognitive recognition of unknown inputs.