Editor’s Spotlight/Take 5: Few FDA Approved AI/ML Orthopaedic Devices Have EU MDR Equivalents or Peer-Reviewed Validation

Iain Feeley FRCS(Tr & Ortho)Buzz about artificial intelligence (AI) may be the only thing we seem to hear about now, but the hype is nothing new. Sixty years ago, pioneers in the field genuinely believed that AI would be ubiquitous before the end of the 20th century. “Within a generation, the problem of artificial intelligence will be substantially solved,” Marvin Minsky famously wrote back in 1967 [7]. He was wrong. In 2002, futurist Ray Kurzweil publicly bet that AI would “surpass native human intelligence” by 2029 [6]. Unless he gets very lucky over the next 3 years, he’s going to lose that bet. Former Google CEO Eric Schmidt prophesied that AI would fix climate change, poverty, war, and cancer [6]. Sundar Pichai, the current CEO of Alphabet (Google’s parent company), thought AI was a bigger deal than fire or electricity in 2018 [3]. The list goes on. Yet these unfulfilled visionary promises haven’t stopped the work on applying AI to healthcare. The results, however, have not been good. A cautionary tale can be found in IBM’s venture with flagship cancer center MD Anderson and its implementation of the Watson-powered “Oncology Expert Advisor” in 2012. By 2016, an audit by the University of Texas revealed that the project had failed to treat a single patient despite a cost of USD 62 million. IBM finally sold off Watson in 2022, having lost USD 3 billion [1, 5]. These are not small amounts of money. And in a world of finite resources, the time, effort, and money spent on things that don’t work represent a serious opportunity cost, since they can’t be used on things that actually work. Enter the regulators. Most countries or regions have a government agency that oversees approval and use of treatments and devices used for medical care. In the United States (US), it’s the Food and Drug Administration (FDA); for the European Union (EU), it’s the Medical Device Regulation (MDR). These responsible entities govern the clinical investigation, production, distribution, and postmarket surveillance of medical devices intended for human use. Perhaps they might have solutions for medical AI technologies that don’t work as advertised. But the FDA—and agencies like it—have somewhat contradictory roles. On the one hand, they are “responsible for protecting the public health by assuring the safety, efficacy, and security of … medical devices,” while on the other, they are “responsible for advancing the public health by helping to speed innovations … and helping the public get accurate, science-based information” [11]. We can have exhaustive trials, costing a great deal of money and time, and risk stifling innovation. Or, we can speed innovation along by assuming that a device which looks reasonably similar to one already on the market is safe, allowing manufacturers to shortcut the process of animal studies followed by phase I, phase II, and phase III human studies before their device can be used by clinicians and patients [10]. Relying on equivalence to devices that are already on the market goes something like this: “Here’s a new intramedullary nail. We’re taking it to market because we think it’s easier/more versatile/stronger than existing devices, but it’s basically the same type of implant.” Of course, if this proves to be untrue, there is a substantial risk of harm. But in the real world, this latter approach—known as the 510(k) pathway [10]—is the one most commonly used for orthopaedic devices, which account for nearly 20% of all devices on the market. Every year, more than 600 novel orthopaedic devices are cleared or approved by the FDA, and 97% of them go through this pathway [4]. (It’s important to note that new drugs are held to a more stringent standard. You can’t put a new statin out there and claim that, because it’s the same as other statins, you don’t need to conduct more studies.) So, is anyone minding the store when it comes to AI in the field of orthopaedic devices? At least one group is, and that group’s work is in the spotlight in this month’s Clinical Orthopaedics and Related Research®. In “Few FDA Approved AI/ML Orthopaedic Devices Have EU MDR Equivalents or Peer-Reviewed Validation” [2], Dr. Aisling Bracken and colleagues explore how regulators are addressing the new technologies of AI and machine learning (ML) by focusing on orthopaedic devices. As she and her coauthors note, these applications “are increasingly embedded in medical technologies, driving applications such as automated clinical documentation, electronic health record integration, diagnostic imaging, and predictive algorithms.” But clinicians and patients need to know whether they are useful, effective, and—above all—safe. What Dr. Bracken and her group have found is that we don’t really know if they are safe, and neither do those responsible for regulation. Looking back at what people worried about in decades past can be a source of fun and schadenfreude, and it may be that future readers will look back at the issues raised by Bracken et al. [2] and think, “how quaint!” Perhaps we’ll have AI in daily orthopaedic practice that works predictably, learns without prompting, and has a clear pathway for regulation and innovation. But until that day, clinicians should approach AI as they might approach any other novel treatment: with caution, and with the understanding that just because a device has been approved does not mean it needs to be embraced. Join me now as I dive deeper with Iain Feeley FRCS(Tr & Ortho) of Midlands Regional Hospital Tullamore in Offaly, Ireland, who is the senior author of “Few FDA Approved AI/ML Orthopaedic Devices Have EU MDR Equivalents or Peer-Reviewed Validation” [2]. Take 5 Interview with Dr. Iain Feeley, senior author of “Few FDA Approved AI/ML Orthopaedic Devices Have EU MDR Equivalents or Peer-Reviewed Validation” Paul A. Manner MD, FRCSC:You noted that “fewer than one-third of FDA approved AI/ML–enabled orthopaedic devices had any identifiable peer-reviewed evidence evaluating their performance” [2]. Is this unusual? How does this compare to standard orthopaedic implants—things that are used in arthroplasty or trauma, for example—as well as to nonorthopaedic devices? Iain Feeley FRCS(Tr & Ortho): It’s not as unusual as it might first appear. Orthopaedic surgery has a long history of implantable devices, both in the US and in Europe, entering practice with relatively limited premarket clinical evidence, as permitted by regulatory, equivalence-based pathways. For arthroplasty and trauma implants, safety and performance have traditionally been evaluated over time through national registries and postmarket surveillance databases. But formal postmarket clinical studies have often been limited, making registries the practical mechanism for detecting real-world performance concerns. AI/ML-enabled devices differ in important ways: Their performance can be context-dependent, evolve over time, and be influenced by local data, workflows, and user behavior. As a result, they are likely to require different approaches to postmarket surveillance than traditional implants. Relying on historical data for an evolving device would clearly be unreliable, and the regulatory challenge therefore becomes establishing monitoring frameworks that are as effective for software-based technologies as registries have been for implants. If this is not in place, the approval for an adaptive system is predicated upon the projected direction of adaptation rather than the actual evolution that occurs. Dr. Manner:You found very few instances where manufacturers used the FDAde novopathway [2]. Does the EU have a similar pathway? And what are the hurdles that a manufacturer might face on this path? Dr. Feeley: Of the 37 AI/ML-enabled orthopaedic devices identified in our study [2], all were cleared through the FDA’s 510(k) pathway, and analysis of their predicate lineages identified a single device that had originally been authorized via the de novo route. This mirrors the wider orthopaedic landscape, where manufacturers overwhelmingly favor the 510(k) pathway because it is faster, more predictable, and better aligned with incremental innovation. Under the EU MDR, genuinely novel devices without a clear comparator typically face a full conformity assessment with a notified body, often at higher risk classification, rather than a dedicated innovation route. These pathways tend to involve greater clinical evidence requirements, longer timelines, and more uncertainty. Taken together, this provides some explanation as to why manufacturers in both the US and EU most often seek to justify innovative technologies, including AI/ML devices, through existing equivalence-based frameworks. Dr. Manner:One concern you raised is that the “predicate devices” used for approval may have very little to do with the new, presumably better, device [2]. In another study, Muehlematter and colleagues [8] noted that a device cleared to assist radiologists in the assessment and characterization of breast abnormalities by use of MRI data served as a predicate device for three rather different devices: one that assessed and characterized brain tissue abnormalities by use of CT image data, another that looked at incidentally detected pulmonary nodules by use of CT image data, and a third that served as an adjunct to diagnostic stress echocardiography for patients undergoing assessment for coronary artery disease. Is this type of predicate appropriate and justified? If not, how does this get through the regulatory process? Dr. Feeley: In a strict regulatory sense, this type of predicate use is permissible, but its appropriateness is more debatable. Under the FDA’s 510(k) framework, substantial equivalence is defined by similarity in intended use and general technological characteristics, or by demonstrating that differing technologies do not raise new questions of safety or effectiveness. As your example illustrates, that definition allows for considerable flexibility, as the predicate device differs substantially in modality, clinical task, anatomy, and clinical urgency. That flexibility helps explain how these clearances occur. Consistent with the findings of Muehlematter and colleagues [8], our study [2] showed that AI/ML predicate networks can span multiple generations, in which the underlying task, data modality, or clinical application changes. This is a pattern often described as “predicate creep.” In practice, AI/ML-enabled devices are frequently cleared using predicate devices that do not incorporate machine learning or other AI technologies, with equivalence argued at a high functional level. If regulatory bodies were to respond to your questions, they would likely argue that using this pathway is justifiable because, if there were no ways to expedite approval, technological advances of medical devices would be severely hampered with unnecessary duplication of processes causing delay. Given the rate of advancement in the field of AI, the potential benefits to healthcare that could accrue through using this technology would be lost. Should this process be revised in order to keep pace with technology—given the volume of new devices that are being brought to market—we might see the intriguing development of an AI-assisted regulator assessing AI medical devices prior to their market debuts. Dr. Manner:If AI/ML devices are changing because of retraining on new datasets or because of user feedback, does that mean they’re changing their interpretations as they’re used more? What does this mean for patient safety? Should the FDA/EU allow upgrades to occur in real time, or should each upgrade be considered a new device? Dr. Feeley: In theory, yes, if an AI/ML system were retrained on new data or user feedback after deployment, its outputs could change over time, meaning the same input might not always produce the same interpretation. That’s both the appeal and the risk; performance may improve, but it could also drift away from what was originally validated, with clear implications for patient safety. In practice, most regulated AI devices today are authorized as locked models. In the US, the FDA has been explicit that all devices cleared so far are static at the point of approval, with meaningful algorithm changes requiring regulatory review. In the EU, there is no single statement confirming that all approved AI devices are locked, and the lack of a central AI device registry makes this difficult to verify. However, MDR change-control requirements and the AI Act’s concept of “substantial modification” effectively push manufacturers toward controlled, versioned updates rather than continuous learning. The solution likely isn’t unrestricted real-time adaptation, nor is it treating every update as a completely new device. The more realistic approach is risk-based, lifecycle oversight. The FDA’s proposed Predetermined Change Control Plan (PCCP) framework reflects this [9], where anticipated updates and monitoring are defined in advance, allowing system modification within clear boundaries. Updates that materially affect clinical performance or patient risk should still trigger closer regulatory scrutiny, although how these guardrails work in practice will be fascinating to follow. Dr. Manner:What happens when an AI/ML enabled device screws up? How would a clinician know, and how would regulatory agencies know? Are existing reporting mechanisms such as the Medical Device Reporting adequate? Dr. Feeley: Unlike overt mechanical failure of an implant, failures in AI systems may not be as obvious. These tools can generate outputs that appear plausible even when they are incorrect. A clinician may only recognize a problem when the output conflicts with their own judgment or with subsequent clinical findings. From a regulatory perspective, agencies currently rely on self-reporting, mainly by manufacturers. In the US, Medical Device Reporting depends on manufacturers reporting adverse events and malfunctions as they become aware of them, and the MDR vigilance system in Europe is similarly led by manufacturers. While important, these systems are passive by design and not well suited to detecting gradual performance drift or systematic bias in AI systems. The EU AI Act strengthens expectations around postmarket monitoring in the EU by requiring manufacturers of high-risk AI systems such as medical devices to implement active postmarket monitoring, logging, and risk management processes. However, it doesn’t replace existing vigilance systems or create an independent, real-time surveillance infrastructure. Academic medical professionals will likely play a large role in the assessment of new devices. Prospective cohort studies or observational data will form a cornerstone to the policing of AI systems. Establishment of registries, in conjunction with regulatory bodies, should be driven by medical and surgical associations such as the American Academy of Orthopaedic Surgeons (AAOS).