Key result
Machine learning and traditional statistical approaches for medical device surveillance of dual-chamber ICDs identified important safety signals but lacked exact agreement (kappa -0.042 to 0.068).
Why the study?
Does a machine learning-based statistical approach complement traditional analytic methods for safety-signal detection in patients receiving dual-chamber ICDs?
Observational (n=71,948)
Yes
Does a machine learning-based statistical approach complement traditional analytic methods for safety-signal detection in patients receiving dual-chamber ICDs?
Machine learning methods identified important safety signals for ICDs but lacked exact agreement with traditional methods, suggesting ensemble approaches may be necessary for comprehensive medical device surveillance.
Suggests ensemble approaches may be necessary for comprehensive ICD safety surveillance; leaves open the optimal integration of machine.
Background: Machine learning methods may complement traditional analytic methods for medical device surveillance. Methods and results: Using data from the National Cardiovascular Data Registry for implantable cardioverter–defibrillators (ICDs) linked to Medicare administrative claims for longitudinal follow-up, we applied three statistical approaches to safety-signal detection for commonly used dual-chamber ICDs that used two propensity score (PS) models: one specified by subject-matter experts (PS-SME), and the other one by machine learning-based selection (PS-ML). The first approach used PS-SME and cumulative incidence (time-to-event), the second approach used PS-SME and cumulative risk (Data Extraction and Longitudinal Trend Analysis [DELTA]), and the third approach used PS-ML and cumulative risk (embedded feature selection). Safety-signal surveillance was conducted for eleven dual-chamber ICD models implanted at least 2,000 times over 3 years. Between 2006 and 2010, there were 71,948 Medicare fee-for-service beneficiaries who received dual-chamber ICDs. Cumulative device-specific unadjusted 3-year event rates varied for three surveyed safety signals: death from any cause, 12.8%–20.9%; nonfatal ICD-related adverse events, 19.3%–26.3%; and death from any cause or nonfatal ICD-related adverse event, 27.1%–37.6%. Agreement among safety signals detected/not detected between the time-to-event and DELTA approaches was 90.9% (360 of 396, k =0.068), between the time-to-event and embedded feature-selection approaches was 91.7% (363 of 396, k =–0.028), and between the DELTA and embedded feature selection approaches was 88.1% (349 of 396, k =–0.042). Conclusion: Three statistical approaches, including one machine learning method, identified important safety signals, but without exact agreement. Ensemble methods may be needed to detect all safety signals for further evaluation during medical device surveillance.
No takes yet. Share an insight, caveat, or question.
Ross et al. (2017) conducted an observational in Patients receiving dual-chamber implantable cardioverter-defibrillators (ICDs) (n=71,948). Machine learning-based propensity score models (PS-ML) vs. Traditional subject-matter expert propensity score models (PS-SME) was evaluated on Agreement among safety signals detected/not detected between statistical approaches. Machine learning and traditional statistical approaches for medical device surveillance of dual-chamber ICDs identified important safety signals but lacked exact agreement (kappa -0.042 to 0.068).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: