CORR Insights®: Can Machine-learning Techniques Be Used for 5-year Survival Prediction of Patients With Chondrosarcoma?

Where Are We Now? Can machine-learning techniques predict 5-year survival in chondrosarcoma? Yes. But what is machine learning and how does it differ from artificial intelligence (AI)? Machine learning is a subset of AI. While AI is driven by software and hardware to mimic defined human activities and behaviors, machine learning uses large amounts of data and algorithms to identify an item, phenomenon, or outcome. It learns from its mistakes, perfecting the identification process [11]. Machine learning is in its infancy in medicine, surgery, and orthopaedics. There is great need for the knowledge it can yield, yet tepid professional enthusiasm, and deficient funding for research in medical applications hamper the adoption of this powerful approach for prediction and decision making. For example, more than 1000 cardiovascular predictive models have been developed, but few are used because they don’t ask the relevant questions and have poor stability of calibration across different populations [12, 16]. (The current chondrosarcoma model hasn’t yet been validated in another population [14].) Inspiration from the orthopaedic and general medical communities is needed to identify the relevant questions where we have the requisite data, and machine learning can help. The AI experts can synthesize results effectively using reductionist and probabilistic models to mimic or exceed the human decision-making process [13]. This will convert skeptics and encourage academic and financial investment in the approach. Why is machine learning needed? Retrospective analysis of cancer survival is a cornerstone of musculoskeletal oncology because the outcome is relevant, binary, and incontrovertible. Data entered may be deficient or disputed, but the result is not. Our current approach uses data from case series that are subject to all biases associated with self-reporting on convenience samples, in particular small-sample bias owing to the rarity of sarcomas [8]. These flaws can be partially overcome in large (nationwide) studies of pediatric cancers like osteosarcoma, Ewing’s sarcoma, and rhabdomyosarcoma where most patients are prospectively-enrolled in the Children’s Oncology Group protocols. There is no such registry or study of adult cancers like chondrosarcoma. Systematic data retrieval often is absent in these small cohort studies and followup can be haphazard. Even when accepted statistical methods like Kaplan-Meier survival analysis are used, there are serious deficiencies like assuming equivalence of censored and lost to followup patients. Bayesian and competing-risk analysis can compensate for some problems like missing data, and inter-relatedness of mortality and the study question, but can’t remedy all the limitations. Machine learning holds the prospect of being a better strategy to analyze diseases like chondrosarcoma. Sophisticated advances in machine learning such as using random forest models versus the more rudimentary tree models or even the “boosted decision tree models” used by Thio and colleagues in the current study [14] highlight the potential for welcome improvements in our understanding of chondrosarcoma and other diseases. Data quality and quantity are major deficiencies limiting what we can do with machine learning at present. As the expression goes, “garbage in, garbage out.” There is little existing population-based data and followup varies widely. The large administrative databases lack sufficient detail to answer clinically relevant questions or guide decision making [5]. Data mining of registries rarely generates insights that experienced clinicians don’t already know. Machine learning can improve the yield of our analyses if we give the machine proper grist for the mill. Chondrosarcoma is an ideal subject for machine learning if we can get good data to interrogate. The methods developed may then be applied throughout orthopaedics. Where Do We Need To Go? We need to standardize the variables we use. This starts with defining the diagnostic criteria for each disease name, grade, and extent. For example, the current chondrosarcoma analysis uses a three-part system to grade tumor “differentiation”, whereas most pathologists consider differentiation as just one factor in assessing tumor grade [2]. In distinction, most prognostication analyses use the Musculoskeletal Tumor Society approach of a two-part system [7]. Even more contentious may be the classification of soft-tissue sarcomas [2]. Consensus needs to be reached on how to reconcile these systems and integrate them with the National Cancer Center Network Guidelines for use in the era of machine learning and artificial intelligence [1]. Developing a web-based application simplifies the process too much by obscuring the sophistication underlying the algorithm. The system also needs to be more user-friendly and avoid ambiguity such as forcing the user to make undefined categorical choices like “local” versus “locally invasive”. A great deal of additional development is needed. Wider usage of the data and analyses should be anticipated. The application developers state that they intend the machine learning projections to be for healthcare providers, but one can easily predict a much wider audience for this information. Consumer-directed transparency (and marketing) is a thrust by patient advocates, the lay public, and pharmaceutical companies [12]. Insurance companies (and lawyers) also seek diagnostic and prognostic information as I discovered in the extensive correspondence I received regarding “The Cancer Weapon We Need Most”, a column I wrote in Reader’s Digest [6] calling for passage of the federal legislation that expanded the US cancer registries used to generate the database for the current study. Early generation machine learned analyses need to be refined substantially before they are ready for widespread use. There is little doubt that such analyses will be invaluable educational tools for the profession and the public. How Do We Get There? The first two problems, lack of definitions and their incorporation into more sophisticated model development requires achieving consensus. The National Comprehensive Cancer Network panel on Bone Tumors, chaired by J. Sybil Biermann, is an excellent, existing panel to address the issue as part of their tasks at the next annual meeting of the group. The panel also includes one of the investigators from the Thio and colleagues study [14] and representatives from all the major cancer centers in the country. Second, refinement of the web-based application should include outside expert opinion. A major requirement is to have external validation on other national and international datasets, as has been done for PathFx, an open-source neural network to predict survival in patients undergoing surgery for pathological fractures [4, 9, 10]. Other disease-specific mechanical learning tools would benefit from including potential stake holders such as representatives of the lay public, nurse educators, and insurers. Finally, the long-range goal should be to use machine learning and AI to direct decision making. All models should be stored and readily accessible as they are for cardiac modeling at the Tufts PACE Center [15]. Blending reductive and probabilistic models may create neural networks that synergize the 30-year-old fundamental mathematics of backprop that underlie deep machine learning with emerging intuitive artificial intelligence [13]. This will resolve any remaining skepticism about the value of mechanical learning in orthopaedic oncology. As stated by futurist Arthur C. Clarke’s third law, “Any sufficiently advanced technology is indistinguishable from magic” [3].