On the Importance of Gradients for Detecting Distributional Shifts in the Wild

Detecting out-of-distribution (OOD) data has become a critical component in the safe deployment of machine learning models in the real world. OOD detection approaches primarily rely on the output or feature space deriving OOD scores, while largely overlooking information from the space. In this paper, we present GradNorm, a simple and effective for detecting OOD inputs by utilizing information extracted from the space. GradNorm directly employs the vector norm of gradients, from the KL divergence between the softmax output and a uniform distribution. Our key idea is that the magnitude of gradients is for in-distribution (ID) data than that for OOD data, making it for OOD detection. GradNorm demonstrates superior performance, the average FPR95 by up to 16. 33% compared to the previous best.