Penalized likelihood approach to estimate a smooth mean curve on longitudinal data

This paper aims to propose a penalized likelihood approach to estimate a smooth mean curve for the evolution with time of a Gaussian variable taking into account the correlation structure of longitudinal data. The model is an extension of the mixed effects linear model including an unspecified function of time f(t). The estimator (circumflex)f(t) is defined as the solution of the maximization of the penalized likelihood and is approximated on a basis of cubic M-spline with a reduced number of knots. We present modifications of four criteria (cross-validation, generalized cross-validation, T of Rice, Akaike's criterion) to estimate the smoothing parameter when data are correlated; these four criteria gave very similar results in the simulation study. The simulation study showed also the superiority of the Bayesian confidence bands of the mean curve over the frequentist ones. We develop empirical Bayes estimates of subject-specific deviations. This approach was applied to study the progression of CD4+ lymphocyte counts in a cohort of HIV patients treated with protease inhibitors.