Monitoring intervention fidelity of a lifestyle behavioral intervention delivered through telehealth

BACKGROUND: Technology-based lifestyle behavioral interventions (i.e., telehealth, mHealth, eHealth, and/or digital health) are becoming an alternative standard of care and possess several advantages over traditional clinical settings such as convenience, cost, and the ability to tailor plans and feedback to a participant's individual needs. These technology-based interventions also present unique challenges to intervention fidelity due to extra elements involved in executing the intervention. Intervention fidelity monitoring is essential to ensure internal and external validity, yet the development and utilization of fidelity protocols is under-reported in the literature. The purpose of this paper is to describe the intervention fidelity protocol for the 24-START study, a behavior change intervention delivered through telephone and internet. This paper also discusses the results of a pilot audit conducted to determine the feasibility of monitoring adherence to the fidelity protocol. METHODS: The 24-START fidelity protocol was developed in accordance with the five fidelity areas outlined by the NIH Behavior Change Consortium (NIH BCC) including: design of study, provider training, delivery of treatment, receipt of treatment, and enactment of treatment. The fidelity strategies provided by the NIH BCC in each area were tailored to fit the specific design of the 24-START study. Twenty-six total fidelity strategies were developed in accordance with the five areas and a corresponding fidelity monitoring plan was created. Because these strategies are only beneficial if implemented, the fidelity monitoring plan was developed to ensure the fidelity strategies are consistently implemented over the course of the intervention. RESULTS: A pilot audit of nine participant files was conducted to test the feasibility of the fidelity protocol developed. Out of the nine participant files reviewed, 89% of scheduled phone calls between a telehealth coach and participant were successfully completed. Of the completed calls, telehealth coaches delivered the intervention as intended 85.3% of the time, and 74% of planned secondary contacts made through the internet were delivered successfully. Additionally, between treatment group dosing was found to be equal. Several weak areas in the fidelity protocol were identified for improvement. The results were satisfactory and the audit was deemed feasible for ongoing use. CONCLUSIONS: The NIH BCC provides a valuable framework for telehealth interventions to develop fidelity protocols ultimately contributing to improved internal and external validity, better translation of results, increased transparency, and increased opportunities for replication within the field. The 24-START pilot audit found the fidelity protocol efficacious and feasible while also identifying areas of weakness in need of revision. The refined protocol will continue to be utilized throughout the data collection phase. Future telehealth interventions should develop and disclose fidelity protocols to improve the overall quality and standard of telehealth interventions.