Fractal basin geometry and the limits to predictability and reproducibility in deep learning

Fundamental limits to predictability are central to our understanding of many physical and computational systems. In deep learning, training outcomes are often difficult to predict and reproduce even when extrinsic sources of randomness are controlled, yet the underlying mechanism remains unclear. Here we propose riddled basin geometry as a unifying dynamical systems mechanism: every initialization leading to one solution lies arbitrarily close to initializations leading to different solutions, forming a fat fractal with structure at arbitrarily fine scales. Using a minimal neural network model, we analytically establish the classical route to riddling and derive a near-zero uncertainty exponent: even sharp increases in initialization precision yield only marginal gains in outcome predictability. In deep networks, intertwined training destinations and near-zero parameter-space exponents provide robust signatures consistent with riddling across the tested architectures, optimizers and training protocols, offering a common dynamical account of otherwise disparate empirical observations. We further introduce functional uncertainty exponents, extending these diagnostics of scale-insensitive irreproducibility from parameter space to the learned functions themselves. Together, our results support riddled basin geometry as a candidate organizing principle for irreproducibility in deep learning, with practical implications for optimization and the reliable deployment of artificial intelligence.