Penalized maximum likelihood estimation in logistic regression and discrimination

Maximum likelihood estimation of the parameters of the binary logistic regression model for pr(H|x) is discussed with separate discussion of sampling from (i) the conditional distribution of H given x, (ii) the joint distribution of H and x, and (iii) the conditional distribution of x given H. Difficulties associated with continuous x in the latter sampling scheme are discussed. To avoid these, penalized maximum likelihood estimates are introduced, which give estimates of the logistic parameters and a nonparametric spline estimate of the marginal distribution of x. Extensions to multinomial logistic regression are outlined.