Key result
A wearable exercise biofeedback system showed a reduction in overall accuracy from >94% in lab-based cross-validation to >75% in healthy participants and >59% in a clinical cohort.
Why the study?
Most wearable exercise biofeedback machine learning models are validated using lab-based cross-validation, which may not reflect real-world performance expected by patients and clinicians.
Does real-world clinical testing reduce the accuracy of wearable sensor-based exercise biofeedback systems compared to lab-based validation?
Does real-world clinical testing reduce the accuracy of wearable sensor-based exercise biofeedback systems compared to lab-based validation?
Real-world clinical validation of wearable exercise biofeedback systems reveals significantly lower accuracy than lab-based testing, highlighting the need for context-specific evaluation.
Answers are grounded in this paper and cite their sources.
Real-world accuracy of wearable biofeedback drops sharply; leaves open the need for context-specific validation before clinical use.
Machine learning models are being utilized to provide wearable sensor-based exercise biofeedback to patients undertaking physical therapy. However, most systems are validated at a technical level using lab-based cross validation approaches. These results do not necessarily reflect the performance levels that patients and clinicians can expect in the real-world environment. This study aimed to conduct a thorough evaluation of an example wearable exercise biofeedback system from laboratory testing through to clinical validation in the target setting, illustrating the importance of context when validating such systems. Each of the various components of the system were evaluated independently, and then in combination as the system is designed to be deployed. The results show a reduction in overall system accuracy between lab-based cross validation (>94%), testing on healthy participants (n = 10) in the target setting (>75%), through to test data collected from the clinical cohort (n = 11) (>59%). This study illustrates that the reliance on lab-based validation approaches may be misleading key stakeholders in the inertial sensor-based exercise biofeedback sector, makes recommendations for clinicians, developers and researchers, and discusses factors that may influence system performance at each stage of evaluation.
No takes yet. Share an insight, caveat, or question.
Argent et al. (2021) studied Physical therapy (n=21). Wearable exercise biofeedback system vs. Lab-based cross validation was evaluated on Overall system accuracy. A wearable exercise biofeedback system showed a reduction in overall accuracy from >94% in lab-based cross-validation to >75% in healthy participants and >59% in a clinical cohort.
Synapse has enriched 10 related papers. Consider them for comparative context: