Mathematical modeling of an optimal control problem for combined chemotherapy and anti-angiogenic cancer treatment protocols

We formulate and analyze an optimal control problem for combined chemotherapy and anti-angiogenic therapy. The model couples tumor burden, vascular support, and a logistic surrogate for healthy tissue that encodes homeostasis and drug-induced depletion. The cost functional balances tumor reduction and drug sparing with toxicity mitigation: Beyond terminal terms and quadratic control regularization, it includes a trajectory reward for healthy tissue and a smooth, differentiable below-threshold penalty based on a softplus construction. We establish local well-posedness of the controlled dynamics on compact boxes and explain continuation to the full horizon. Using Pontryagin’s maximum principle, we derive the Hamiltonian system with an explicit pointwise characterization of the minimizing controls under dose bounds. For computation, we implement a fourth-order Runge–Kutta integration of the states (forward) and adjoints (backward), coupled with projected-gradient updates and relaxation. Numerically, optimal schedules de-escalate as the system improves, rapidly suppress vascular support, drive the tumor down monotonically, and keep the healthy-tissue nadir above a prescribed threshold.