Measuring fidelity of brief cognitive behavior therapy for insomnia: Development, reliability and validity

Measuring fidelity is critical in program evaluations to assess how implementation influences outcomes. Implementer fidelity is comprised of adherence to the treatment content and competence of treatment delivery. Cognitive behavioral therapy for insomnia (CBT-I) is well-established and empirically supported with variants such as brief CBTi (bCBTi) showing promise in reducing both insomnia and depression. The impact of therapist fidelity on treatment outcomes is unknown in part because reliable measures have not been available. We developed measures of therapist fidelity for bCBT-i and assessed the impact of fidelity in the context of a pilot study with veterans in primary care. Audio recordings from 23 participants (78% male) were coded. Therapist adherence measures were created for each session along with a single measure of therapist competence. Inter-rater reliability was established and predictive validity was determined. For total adherence, inter-rater reliability was excellent across sessions (ICC =.73 -.80). The competence measure showed good reliability across all sessions (ICC =.57) and was internally consistent (Cronbach’s alpha =.75). There was only 10% of shared variance between adherence and competence. All of the fidelity measures demonstrated associations with outcomes in the predicted direction; therapist adherence was significantly associated with decreased depression. • The correlation between therapists’ raw adherence and competence scores was.30 which indicates only 10% of shared variance. Adherence and competence measures clearly captured unique information and separate scales are important to reflect these domains of fidelity. • Both the fidelity measures of adherence and competence showed associations with outcomes of insomnia and depression in the predicted direction. • Therapist adherence was significantly associated with decreased depression.