Hill Space is All You Need

We characterize "Hill Space"-the constraint topology created by W = tanh(Ŵ) ⊙ σ(M ̂), where ⊙ denotes element-wise multiplication-as a systematic framework for discrete selection in neural networks. This constrained parameter space enables mathematical enumeration: optimal weights for discrete operations can be calculated rather than learned, transforming primitive discovery from optimization-dependent exploration into systematic cartography. Hill Space creates stable plateaus where neural networks reliably converge to discrete weight configurations such as [1,1] and [1,-1], enabling precise mathematical operations. We demonstrate this across arithmetic and trigonometric operations, achieving high precision with deterministic convergence. We achieve high precision across arithmetic and trigonometric operations, with models converging deterministically regardless of random seed (±0.5 epoch variance across runs). The constraint topology enables aggressive learning rates and extreme extrapolation with accuracy limited primarily by floating-point precision and activation saturation rather than optimization failures. Our key discovery is that Hill Space enables systematic exploration of discrete selection problems through direct weight manipulation rather than training. We demonstrate this methodology using mathematical operations as a clean test domain, establishing a foundation for neural networks that perform reliable discrete selection through principled constraint design.