Improving Gradient Descent Optimization through Guided Exploration of Loss Landscape

Machine learning is extensively applied in measurement technology, particularly in the domain of autonomous driving. Gradient descent based optimization algorithms, which are a crucial aspect of deep learning models, often suffer poor performance because of prevalence of suboptimal regions in the loss function landscape. This paper presents Guided Exploration (GE) algorithm that combines local and global search strategies to navigate these suboptimal regions by leveraging information about the loss landscape's topology. It employs guided exploration to identify promising regions in the loss function landscape and then circumnavigates these regions, consequently improving convergence and boosting performance. Experiments show an improved accuracy over SGD and Adam algorithms respectively implemented on the benchmark fashion-MNIST dataset.