We present Homeostatic Neural Architecture (HNA), a framework for online Hessian-drivenstructural self-modification of neural networks during a single training run. Unlike NeuralArchitecture Search (NAS), which treats architecture discovery as an external black-boxoptimisation problem requiring thousands of GPU-hours, HNA continuously interrogates theempirical Fisher diagonal to make fine-grained structural decisions — neuron splitting, neuronpruning, layer insertion, layer removal, and skip-connection management — with all operationsbeing zero-disruption: network function is exactly preserved at each structural event.On raw CIFAR-10 pixels ( no dropout, -1,1 normalisation, with augmentation), HNA discovers a10M-parameter architecture (3072→3027→249→10) achieving 65.8% validation accuracy with a3.0% train/val gap — strictly outperforming a human-designed 40M-parameter network (62.3%validation, 33.1% gap) at one quarter the parameter count with an 11× smaller generalisation gap.Even without structural modification, training the HNA-discovered architecture as a fixed networkachieves 64.9% test accuracy, surpassing all human baselines at comparable parameter counts.On MNIST raw pixels HNA achieves 98.6% validation accuracy. Threshold evolution requiresapproximately 3 hours on a single consumer GPU (NVIDIA RTX 4060 8GB) at partial utilisation.
Hassan Muhammad (Sat,) studied this question.