Nonlinear Least Squares Estimation

We are given a set of N responses Yₜ which have arisen from a nonlinear regression model {equation*}{(1.1)}Y_t = f(x_t, θ) + e_t; t = 1, 2, ⋯, N.{equation*} Here xₜ denotes the tth fixed input vector of k elements giving rise to Yₜ, whilst θ is an m-element unknown parameter vector with elements θᵢ and the eₜ are a set of N independent error residuals from N(0, σ²) with σ² unknown. The expectations of the Yₜ, are therefore the functions f(xₜ, θ) which will be assumed to satisfy certain regularity conditions. The problem is to estimate θ notably by least squares. In this paper we shall develop an iterative method of solution of the least squares equations which has the following properties: (a) the computational procedure is convergent for finite N; (b) the resulting estimators are asymptotically 100# efficient as N → ∞. In Sections 2-4 we give a survey of our results leaving the mathematical proofs to Sections 5-7 whilst in Section 8 we illustrate our method with an example. Although our theoretical development is oriented towards our specific goals certain results are proved in a somewhat more general form. Some of our theory will be seen to correspond to well known theorems on stochastic limits which have to be reproved because of certain modifications which we require.