Simulation study demonstrates penalized maximum likelihood reduces bias and variance in small-sample logit models, suggesting it outperforms standard maximum likelihood without trade-offs.
In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993, Biometrika , 80 , 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps more importantly, greatly reduces the variance of the usual ML estimator. Thus, researchers do not face a bias-variance tradeoff when choosing between the ML and PML estimators—the PML estimator has a smaller bias and a smaller variance. We use Monte Carlo simulations and a re-analysis of George and Epstein (1992, American Political Science Review , 86 , 323–337) to show that the PML estimator offers a substantial improvement in small samples (e.g., 50 observations) and noticeable improvement even in larger samples (e.g., 1000 observations).
No takes yet. Share an insight, caveat, or question.
Rainey et al. (2021) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: