Answers are grounded in this paper and cite their sources.
Tuning hyperparameters such as learning rate decay and stop conditions is typically done by assessing loss on a held-out validation set. This work introduces neural velocity (NeVe), the rate of change in neuron transfer functions, as a novel indicator of model convergence. We leverage NeVe to create a dynamic training method where learning rates and stop conditions are adjusted based on neural velocity computed from an auxiliary dataset generated by sampling noise. Our experiments across multiple tasks and architectures show that our approach performs on par with traditional validation-based methods, outperforming them in some tasks. Our method does not require withholding data for validation, making it especially advantageous in data-limited scenarios. This work highlights the potential of neural velocity as a key metric for optimizing neural network training.
No takes yet. Share an insight, caveat, or question.
Dalmasso et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: