Answers are grounded in this paper and cite their sources.
We present a novel neural network architecture with strong innate sparsity properties that also allows explaining the model decisions by identifying the bases for selecting and rejecting different decision alternatives. The elementary building block of the proposed architecture is a Hill neuron with antagonistic activation outputs, one for activation and the other for repression. Both outputs are relayed to the neurons of the next layer modulated by distinct weights and included jointly in the summation to provide the input to the next layer neurons, and pitted against each other during training that promotes one of the two by increasing its weight and reducing that of the other in a way that minimizes the training error. The activation and repression functions are themselves parameterized by an activation threshold and a Hill coefficient that are also included in the training updates, shaping them actively and adaptively by training. Post-training evaluation of Hill coefficients and connection weights allowed substantial reduction in model size while preserving the recognition performance of the original model without re-training. Experiment results on MNIST datasets of handwritten digits and letters show that the recognition performance of conventional neural network models of the same size featuring single-output artificial neurons can be matched using a small fraction of hidden layer neurons and connections: On the MNIST dataset of handwritten digits, the reduced model possessed %42.75 of the hidden layer neurons and %15.82 of the connections of a conventional artificial neural network. On the EMNIST dataset of handwritten letters, the percentages were %83.38 and %31.65, respectively. Dual saliency maps of activation and repression associated with the output layer neurons along with the important hidden layer neurons highlighted the image features that were instrumental in favoring the selected class and rejecting the others. These results demonstrate the potential of the proposed Hill neural networks as suitable candidates for sparse, interpretable and explainable learning systems.
No takes yet. Share an insight, caveat, or question.
Bilge Karaçalı (2026) studied this question.