Clinical Simulation-based Usability Testing of a Mobile Telestroke System (P3.104)

Objective: To understand perceptions of a mobile ambulance-based telestroke system and evaluate system usability. Background: Recent emphasis has been placed on optimizing prehospital stroke care and calls for innovative approaches to further enhance the paradigm of prehospital stroke care. Mobile prehospital telestroke presents a novel solution to improve stroke diagnosis and reduce treatment times. Design/Methods: An ambulance was equipped with a mobile telemedicine system to perform remote stroke assessments. Scripted scenarios were performed by actors during transport and evaluated by physicians using the NIH stroke scale (NIHSS). Scores obtained during transport were compared with independent bedside and original scripted NIHSS scores. Participants completed the System Usability Scale (SUS), NASA task load index, audio-video quality scale and a modified Acceptability of Technology survey after completing the NIHSS evaluations. In addition, interviews were conducted to evaluate user’s experience and perceptions. Descriptive analysis was used for all surveys. Weighted kappa was used to compare the agreement in NIHSS scores. A regression model was used to further account for variations. Results: Ten scripted scenarios were simulated twice during the mobile transport and once at bedside. All simulations were completed except for one. NIHSS scores between mobile, bedside and original scripted scenarios revealed good agreement [weighted kappa=0.76 (95% CI: 0.63–0.9, p=0.63)]. There were no statistically significant differences in NIHSS scores between evaluations. The results were independent of stroke scenarios, physicians, and actors. Overall, 92% and 81% raters deemed video and audio quality as “good” or “excellent” (rating <3) respectively. The overall mean SUS score was 69.1 (13.3). Content analysis identified strengths, usability issues, and safety concerns. Overall users identified similar usability issues with the technology including: audibility, connectivity, equipment stability, and poor visibility. Conclusions: Our study shows the feasibility and capability of a mobile telestroke system to accurately assess actors simulating stroke patients during transport. Study Supported by: Virginia Commonwealth University (VCU) Dean’s Enhancement Award, VCU CTSA (UL1TR000058) from the National Center for Advancing Sciences, CCTR Endowment Fund of Virginia Commonwealth University (VCU) Disclosure: Dr. Heath has nothing to disclose. Dr. Mehndiratta has nothing to disclose. Dr. Decker has nothing to disclose. Dr. Lawrence has nothing to disclose. Dr. Whitt has nothing to disclose. Dr. Fellows has nothing to disclose. Dr. Broadnax has nothing to disclose. Dr. Lavrentyev has nothing to disclose. Dr. Feldman has nothing to disclose. Dr. Henry has nothing to disclose. Dr. Wong has nothing to disclose. Dr. Ruiz has nothing to disclose. Dr. Hekmatodoost has nothing to disclose. Dr. Ono has nothing to disclose. Dr. Bhatti has nothing to disclose. Dr. Rahim has nothing to disclose. Dr. Toqeer has nothing to disclose. Dr. Ricks has nothing to disclose. Dr. Madu has nothing to disclose. Dr. Blankson has nothing to disclose. Dr. Heath has nothing to disclose. Dr. Brown has nothing to disclose. Dr. Bennam has nothing to disclose. Dr. Lu has nothing to disclose. Dr. Felton has received compensation from Biogen Idec, Teva, Bristol-Myers Squibb, Serono, Sanofi-Aventis and Pfizer for speaking. Dr. Felton has received research support from the following organizations: NIH (WARCEF), Merck (MERCK), Novartis (ACCOMPLISH), Eli Lilly [amp] Centocor (AbESTT), NMT Medical (CLOSURE), Boehringer Ingelheim (PRoFESS), Ono Pharma USA (ONO), Biogen Idec (CHA). Dr. Ornato has nothing to disclose. Dr. Chapman has nothing to disclose.