Applying advanced psychometric approaches yields differential randomised trial effect sizes: Secondary analysis of individual participant data from antidepressant studies using the Hamilton Rating Scale for Depression

ObjectiveAs multiple sophisticated techniques are used to evaluate psychometric scales, in theory reducing error and enhancing measurement of patient reported outcomes, we aimed to determine whether applying different psychometric analyses would demonstrate important differences in treatment effects.Study Design and SettingWe conducted secondary analysis of individual participant data from 20 antidepressant treatment trials obtained from Vivli.org (n=6,843). Pooled item-level data from the HRSD-17 were analysed using confirmatory factory analysis (CFA), item response theory (IRT) and network analysis (NA). Multilevel models were used to analyse differences in trial effects at approximately 8 weeks (range 4-12 weeks) post-treatment commencement, with standardised mean differences calculated as Cohen’s d. Effect size outcomes for the original total depression scores were compared with psychometrically-informed outcomes based on abbreviated and weighted depression scores. ResultsSeveral items performing poorly during psychometric analyses and were eliminated, resulting in different models being obtained for each approach. Treatment effects were modified as follows per psychometric approach: 10.4%-14.9% increase for CFA, 0%-2.9% increase for IRT, 14.9%-16.4% reduction for NA. ConclusionPsychometric analyses differentially moderate effect size outcomes depending on the method used. In a 20-trial sample, factor analytic approaches increased treatment effect sizes relative to the original outcomes, NA decreased them, and IRT results reflected original trial outcomes.