ABSTRACT This study evaluates an automatic differentiation (AD)-assisted first-order conditional estimation (FOCE) algorithm integrated into the FOCE Extended Least Squares (FOCE ELS) framework within Phoenix NLME 8.6. AD is a method commonly used in machine learning to provide exact (to machine precision) values for the partial derivatives of a function. There are two modes of AD, forward mode and reverse mode 1. In the case of nonlinear regression, the objective function of interest is the negative twice the log-likelihood (−2LL), expressed as a function of the model parameter values. Previously released versions of NLME use finite difference (FD), a numerical approximation for the partial derivative. To examine the impact of AD compared to FD on execution speed, we constructed 72 models ranging from one to three compartments. All models included Michaelis-Menten elimination, first-order absorption, and combined additive and proportional residual error models. The models were selected to provide a range of complexity and numerical identifiability, from simple, well-identified models to complex, poorly identifiable ones. We demonstrate that AD improves performance in both the estimation and covariance steps. The AD method had no significant impact on estimation robustness or the success of the covariance step.
Sale et al. (Thu,) studied this question.