Abstract Deep learning has revolutionized computational imaging, yet its real-world deployment remains constrained by two critical challenges: poor generalization under dynamic conditions and the emergence of hallucinatory artifacts. By leveraging a physics-guided framework based on scattering media, a model system where controlled variations in light transmission matrices (T T) isolates these challenges, we unravel the mechanistic interplay between generalization limits and hallucination origins. We demonstrate that a network’s generalization capacity is fundamentally bounded by its ability to accommodate distinct inverse mappings (T^-1 T − 1), while hallucinations arise when this capacity is exceeded, resulting in unconstrained, non-physical predictions. We also identify residual ballistic light, if not negligible, as a stabilizing anchor, enabling robust predictions under scattering variability. Integrating experimental validation with wave-optics simulations, we establish a universal framework that links these phenomena, showing that strategic training on diverse physical mappings enhances generalization while suppressing hallucinations. This work bridges physics-driven interpretability with AI design, offering actionable strategies to develop reliable models for applications ranging from medical imaging through biological tissues to autonomous navigation in scattering environments.
Zhang et al. (Thu,) studied this question.