What question did this study set out to answer?

This research aims to develop a vectorized hypercomplex perturbation method for efficiently calculating exact gradients and Hessians.

April 5, 2026Open Access

One-Pass Exact Hessians via Hypercomplex Perturbation: A Vectorized Implementation with Implicit Layer Applications

Key Points

This research aims to develop a vectorized hypercomplex perturbation method for efficiently calculating exact gradients and Hessians.
Introduced a vectorized hypercomplex perturbation method.
Conducted benchmarks for performance comparisons at different sizes (n=16, 32, 100).
Applied the method to implicit layer functions to recover Hessians accurately.
Compared performance against JAX unrolled and demonstrated faster convergence in trust-region problems.
Achieved speedup of 23× at n=16, 108× at n=32, and 1129× at n=100 compared to earlier versions.
Demonstrated that the hypercomplex method recovers the correct Hessians for implicit layers, outperforming JAX unrolled by 27–403×.
Trust-region demo showed convergence in 16 iterations, highlighting method stability.

Abstract

Version 3.0 — complete manuscript with vectorized implementation, benchmarks, and implicit layer applications. This paper introduces a vectorized hypercomplex perturbation method that recovers exact gradients and full Hessians of scalar-valued functions in a single forward evaluation. Speedup over v0.1.0: 23× at n=16, 108× at n=32, 1129× at n=100. For implicit layers z*=f(z*,θ), recovers the IFT-correct Hessian — JAX unrolled gives a different result. hcderiv is 27–403× faster than JAX unrolled for n=3–8. Trust-region demo: exact and FD converge in 16 iterations; diagonal stalls at f=6.2e-3 after 150. 59 tests pass. HC vs JAX: 4×10⁻¹⁶. GitHub: https://github.com/zetta55byte/hypercomplex | PyPI: https://pypi.org/project/hcderiv/ | Software DOI: 10.5281/zenodo.19389522

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper