Machine-learning prediction models in orthopedic surgery demonstrated a median area under the curve of 0.80, with medical management being the most commonly studied outcome domain.
Systematic Review (n=59)
Machine learning prediction models are increasingly developed for orthopedic surgery, particularly in spinal surgery and medical management, but often lack adequate reporting of calibration and decision-curve analysis.
Effect estimate: Median AUC 0.80
Background and purpose - Advancements in software and hardware have enabled the rise of clinical prediction models based on machine learning (ML) in orthopedic surgery. Given their growing popularity and their likely implementation in clinical practice we evaluated which outcomes these new models have focused on and what methodologies are being employed.Material and methods - We performed a systematic search in PubMed, Embase, and Cochrane Library for studies published up to June 18, 2020. Studies reporting on non-ML prediction models or non-orthopedic outcomes were excluded. After screening 7,138 studies, 59 studies reporting on 77 prediction models were included. We extracted data regarding outcome, study design, and reported performance metrics.Results - Of the 77 identified ML prediction models the most commonly reported outcome domain was medical management (17/77). Spinal surgery was the most commonly involved orthopedic subspecialty (28/77). The most frequently employed algorithm was neural networks (42/77). Median size of datasets was 5,507 (IQR 635-26,364). The median area under the curve (AUC) was 0.80 (IQR 0.73-0.86). Calibration was reported for 26 of the models and 14 provided decision-curve analysis.Interpretation - ML prediction models have been developed for a wide variety of topics in orthopedics. Topics regarding medical management were the most commonly studied. Heterogeneity between studies is based on study size, algorithm, and time-point of outcome. Calibration and decision-curve analysis were generally poorly reported.
Ogink et al. (Thu,) conducted a systematic review in Orthopedic surgical outcomes (n=59). Machine-learning prediction models was evaluated on Model performance (Area under the curve) (Median AUC 0.80). Machine-learning prediction models in orthopedic surgery demonstrated a median area under the curve of 0.80, with medical management being the most commonly studied outcome domain.