What type of study is this?

September 10, 2025Open Access

Adaptive Learning Rate Methods for Complex-Valued Neural Networks

KMKayol S. MayerUniversidade Estadual de Campinas (UNICAMP)JSJonathan Aguiar SoaresUniversidade Estadual de Campinas (UNICAMP)ACAriadne Arrais CruzUniversidade Estadual de Campinas (UNICAMP)

Key Points

Adaptive learning rate methods enhance convergence in complex-valued neural networks, outpacing traditional approaches.
This research compares different adaptive gradient techniques, showing improved mean-squared-error performance for complex-valued neural networks.
Employing optimizers like RMSProp and Nadam, the proposed methods analyze computational complexities in neural network architectures.
The findings suggest the need for advancing learning heuristics in complex-valued neural networks compared to real-valued ones.

Abstract

Artificial neural networks (ANNs) have become a popular tool in digital signal processing (DSP). Among the widespread ANN architectures, complex-valued neural networks (CVNNs) have been extensively studied in image processing and telecommunications. Unlike their real-valued counterparts, CVNNs can handle signals directly in the complex domain. Due to this capability, CVNNs usually exhibit higher accuracy and improved convergence compared to real-valued neural networks (RVNNs). Despite their improved performance in several applications, CVNNs still lag behind RVNNs in terms of learning techniques and heuristics. In this context, we propose adaptive learning rate approaches for CVNNs, extending the well-known adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp), AdaMax, AMSGrad, softplus AMSGrad (SAMSGrad), Nesterov-accelerated adaptive moment estimation (Nadam), and DiffGrad to the complex domain. Computational complexities of the proposed optimizers are analyzed for CVNN architectures. Results are compared in terms of mean-squared-error convergence.

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper