Answers are grounded in this paper and cite their sources.
Analysis shows deep Heaviside networks can improve expressiveness using skip connections or linear neurons, impacting regression accuracy.
We show that deep Heaviside networks (DHNs) have limited expressiveness but that this can be overcome by including either skip connections or neurons with linear activation. We provide upper and lower bounds for the Vapnik-Chervonenkis (VC) dimensions and approximation rates of these network classes. As an application, we derive statistical convergence rates for DHN fits in the nonparametric regression model.
No takes yet. Share an insight, caveat, or question.
Kong et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: