What type of study is this?

This is a Experimental Study study.

September 12, 2025Open Access

A Novel Exponential Continuous Learning Rate Adaption Gradient Descent Optimization Method

Key Points

The proposed method achieves a high success rate and fast convergence with lower reliance on hand-tuned hyper-parameters.
The two variants, C2Min and P2Min, demonstrate significant performance improvements in comparison to existing optimizers such as Adam and Lion.
The optimization method scales linearly with dimension, maintaining rotation invariance to address existing limitations in the field.
Experiments conducted on benchmark data-sets like MNIST and Tiny ImageNet showcase the method's versatility and effectiveness across various applications.

Abstract

We present two novel, fast gradient based optimizer algorithms with dynamic learning rate. The main idea is to adapt the learning rate α by situational awareness, mainly striving for orthogonal neighboring gradients. The method has a high success and fast convergence rate and relies much less on hand-tuned hyper-parameters, providing greater universality. It scales linearly (of order O(n)) with dimension and is rotation invariant, thereby overcoming known limitations. The method is presented in two variants C2Min and P2Min, with slightly different control. Their impressive performance is demonstrated by experiments on several benchmark data-sets (ranging from MNIST to Tiny ImageNet) against the state-of-the-art optimizers Adam and Lion.

Read Full Paperexternally

Bookmark

View Full Paper

Cite This Study

Kleinsorge et al. (Fri,) studied this question.

synapsesocial.com/papers/68d44c5531b076d99fa561c7 https://doi.org/https://doi.org/10.52825/th-wildau-ensp.v2i.2939

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

Bookmark

View Full Paper