Reliability in Adolescent Reporting of Clinician Counseling, Health Care Use, and Health Behaviors

BACKGROUND: Accurate measures of health-care use by adolescents would be useful in managed care quality assurance, public health surveillance, and health-care research. OBJECTIVE: To assess test-retest reliability and factors associated with reliability of adolescent reports of clinician counseling, preventive health services, and health behaviors. RESEARCH DESIGN: A convenience sample of high school students (N = 253) completed identical paper-and-pencil surveys in school and 2 weeks apart. Multiple linear regression was used to evaluate the influence on response reliability of individual factors and question item characteristics. Reliability was assessed using Cohen kappa. RESULTS: Kappa values for specific questions varied widely (0.94-0.33). Median kappa values for behavioral, counseling, and health-service questions were 0.74, 0.63, 0.56, respectively. Lower sentence complexity, certain time frames (ever, age at first occurrence, last time), and behavioral question type were associated with greater reliability in adolescent reporting (final model R2 = 0.54). Adolescents' age and ethnicity were not predictive of reliability, though girls were slightly more reliable reporters than boys. Overall, the prevalence of responses at times 1 and 2 were similar; 95% of responses at time 2 were within 5 percentage points of time-1 estimates (SD = 2.4). CONCLUSIONS: The reliability of adolescent reporting was strongly influenced by question characteristics such as sentence complexity and time frame; these should be carefully considered in the construction of questionnaires for adolescents. Adolescents can be an accurate source of health-care service data.