A Framework for Differentiating Black-Box Convex Optimization Solvers

Optimization is a core engine of scientific computing and decision-making, and its recent integration with deep learning has given rise to differentiable optimization. By embedding constrained optimization problems as implicit layers, one can inject structure and domain knowledge into trainable models. Yet in practice, many existing approaches for differentiating the solution of the optimization problem with respect to its defining parameters often rely on specific integrated solvers. This integration limits their applicability, including their use in neural network architectures and bi-level optimization tasks, restricting users to a narrow selection of solver choices. This dissertation develops a modular, solver-agnostic framework for differentiating optimization solvers by leveraging local geometry at the optimum rather than differentiating through solver internals. The first part, dQP, shows that derivatives for quadratic programs can be recovered by solving a reduced linear system defined by the active constraints, yielding plug-and-play differentiation with first-order correctness. The second part, dOPT, extends this perspective to general convex conic programs through active cone geometries, with concrete formulations for nonlinear programs (NLP), second-order cone programs (SOCP), and semidefinite programs (SDP). Comprehensive benchmark experiments demonstrate practical effectiveness, with robust and scalable performance across diverse problem instances. The third part, CryoSeed, is a distinct line of work on cryo-EM 3D refinement, motivated by the same broader goal of modular and differentiable system design: replacing black-box pipelines with composable, extensible, and GPU-native components that enable efficient computation, flexible method development, and future end-to-end learning.