Answers are grounded in this paper and cite their sources.
Demonstrates efficient optimization in machine learning, suggesting improved techniques for complex problems.
This paper presents a two-dimensional search algorithm for quasi-Newton methods applied to ill-conditioned and degenerate unconstrained optimization problems. At each iteration, the space is decomposed into an orthogonal sum of two subspaces based on the spectral decomposition of the approximate Hessian (updated via the BFGS or SR1 formula) and a regularization parameter. The search direction in one subspace is computed using a quasi-Newton scheme, while an alternative optimization method (e.g., gradient descent or conjugate gradient) is employed in the complementary subspace. The next iterate is obtained by minimizing a fourth-order local model of the objective function in two dimensions with respect to the step-size parameters along these directions. The proposed approach enables efficient handling of spectral degeneracy by combining curvature-aware and gradient-based updates within a unified framework. The efficiency of the proposed method is demonstrated through numerical experiments on standard test problems from unconstrained optimization and machine learning. The results are compared with implementations from widely used software environments, including R, Scilab, Python, and PyTorch.
No takes yet. Share an insight, caveat, or question.
Viktor M. Zadachyn (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: