The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate

For given data (tᵢ ,yᵢ ),i = 1, ⋯ ,m, we consider the least squares fit of nonlinear models of the form \[ η ({ a},{ α} ;t) = ∑ j = 1^n {a_j φ _j ({ α} ;t), { a} ∈ R^n , { α} ∈ R^k .} \] For this purpose we study the minimization of the nonlinear functional \[ r({ a},{ α} ) = ∑i = 1^m {( {y_i - η ( {{ a},{ α} ,t_i } )} )^2 } . \] It is shown that by defining the matrix \ Φ ( α )\ i,j = φ ⱼ ( α ;tᵢ ), and the modified functional r₂ ( α ) = \| y - Φ ( α ) Φ ^ + ( α ) y \|₂², it is possible to optimize first with respect to the parameters α, and then to obtain, a posteriors, the optimal parameters a. The matrix Φ ^ + (α ) is the Moore–Penrose generalized inverse of Φ (α ). We develop formulas for the Frechet derivative of orthogonal projectors associated with Φ (α ) and also for Φ ^ + (α ), under the hypothesis that Φ (α ) is of constant (though not necessarily full) rank. Detailed algorithms are presented which make extensive use of well-known reliable linear least squares techniques, and numerical results and comparisons are given. These results are generalizations of those of H. D. Scolnik [20] and Guttman, Pereyra and Scolnik [9].