Confidence Inversion in Open-Set Underwater Acoustic Target Recognition: A Leakage-Safe Multi-Corpus Pilot Study

Underwater acoustic target recognition (UATR) systems are typically evaluated in a closed label space, whereas real deployments routinely face unseen targets and shifted acquisition conditions. We evaluate five score/loss configurations—ResNet-18 classifiers trained with cross-entropy (CE) or an entropic open-set–style penalty (EOS-L) and scored with maximum softmax probability (MSP), the energy score, or OpenMax—across four corpora and six recording-disjoint protocols (3 seeds each): four within-corpus class-holdout protocols and two cross-corpus settings with domain-shifted unknowns on different corpus pairs. CE+MSP falls below chance-level AUROC (0.34–0.37) on the Oceanship holdout and the QiandaoEar22→Oceanship cross-corpus protocols, showing that in these two evaluated protocols unknown samples receive higher confidence than known ones more often than not. Without additional training, the energy score improves AUROC on the inversion protocols by +0.21 and +0.12 across the three evaluated seeds (nominal paired tests, p ≤ 0.013; exploratory, direction-consistent). OpenMax and EOS-L yield protocol-dependent gains and regressions. Even the best logit-space protocol mean has an unknown mis-acceptance rate of 0.667 at 95% known acceptance. Post-hoc feature-space scoring on the same frozen encoders behaves differently: a ViM-style residual score attains 0.934 and 0.939 AUROC on the two cross-corpus protocols (FPR@95TPR 0.205 and 0.405) but only 0.640 on the Oceanship within-corpus holdout—separability that conventional logit-space confidence fails to capture. This separability may reflect acquisition-domain differences rather than semantic vessel novelty: feature-space residual scoring substantially improves detection of the composite cross-corpus unknowns evaluated here, whereas within-corpus novel-class rejection remains weak under the main no-background configurations.