Iain Feeley FRCS(Tr for the European Union (EU), it’s the Medical Device Regulation (MDR). These responsible entities govern the clinical investigation, production, distribution, and postmarket surveillance of medical devices intended for human use. Perhaps they might have solutions for medical AI technologies that don’t work as advertised. But the FDA—and agencies like it—have somewhat contradictory roles. On the one hand, they are “responsible for protecting the public health by assuring the safety, efficacy, and security of … medical devices,” while on the other, they are “responsible for advancing the public health by helping to speed innovations … and helping the public get accurate, science-based information” 11. We can have exhaustive trials, costing a great deal of money and time, and risk stifling innovation. Or, we can speed innovation along by assuming that a device which looks reasonably similar to one already on the market is safe, allowing manufacturers to shortcut the process of animal studies followed by phase I, phase II, and phase III human studies before their device can be used by clinicians and patients 10. Relying on equivalence to devices that are already on the market goes something like this: “Here’s a new intramedullary nail. We’re taking it to market because we think it’s easier/more versatile/stronger than existing devices, but it’s basically the same type of implant.” Of course, if this proves to be untrue, there is a substantial risk of harm. But in the real world, this latter approach—known as the 510(k) pathway 10—is the one most commonly used for orthopaedic devices, which account for nearly 20% of all devices on the market. Every year, more than 600 novel orthopaedic devices are cleared or approved by the FDA, and 97% of them go through this pathway 4. (It’s important to note that new drugs are held to a more stringent standard. You can’t put a new statin out there and claim that, because it’s the same as other statins, you don’t need to conduct more studies.) So, is anyone minding the store when it comes to AI in the field of orthopaedic devices? At least one group is, and that group’s work is in the spotlight in this month’s Clinical Orthopaedics and Related Research®. In “Few FDA Approved AI/ML Orthopaedic Devices Have EU MDR Equivalents or Peer-Reviewed Validation” 2, Dr. Aisling Bracken and colleagues explore how regulators are addressing the new technologies of AI and machine learning (ML) by focusing on orthopaedic devices. As she and her coauthors note, these applications “are increasingly embedded in medical technologies, driving applications such as automated clinical documentation, electronic health record integration, diagnostic imaging, and predictive algorithms.” But clinicians and patients need to know whether they are useful, effective, and—above all—safe. What Dr. Bracken and her group have found is that we don’t really know if they are safe, and neither do those responsible for regulation. Looking back at what people worried about in decades past can be a source of fun and schadenfreude, and it may be that future readers will look back at the issues raised by Bracken et al. 2 and think, “how quaint!” Perhaps we’ll have AI in daily orthopaedic practice that works predictably, learns without prompting, and has a clear pathway for regulation and innovation. But until that day, clinicians should approach AI as they might approach any other novel treatment: with caution, and with the understanding that just because a device has been approved does not mean it needs to be embraced. Join me now as I dive deeper with Iain Feeley FRCS(Tr performance may improve, but it could also drift away from what was originally validated, with clear implications for patient safety. In practice, most regulated AI devices today are authorized as locked models. In the US, the FDA has been explicit that all devices cleared so far are static at the point of approval, with meaningful algorithm changes requiring regulatory review. In the EU, there is no single statement confirming that all approved AI devices are locked, and the lack of a central AI device registry makes this difficult to verify. However, MDR change-control requirements and the AI Act’s concept of “substantial modification” effectively push manufacturers toward controlled, versioned updates rather than continuous learning. The solution likely isn’t unrestricted real-time adaptation, nor is it treating every update as a completely new device. The more realistic approach is risk-based, lifecycle oversight. The FDA’s proposed Predetermined Change Control Plan (PCCP) framework reflects this 9, where anticipated updates and monitoring are defined in advance, allowing system modification within clear boundaries. Updates that materially affect clinical performance or patient risk should still trigger closer regulatory scrutiny, although how these guardrails work in practice will be fascinating to follow. Dr. Manner:What happens when an AI/ML enabled device screws up? How would a clinician know, and how would regulatory agencies know? Are existing reporting mechanisms such as the Medical Device Reporting adequate? Dr. Feeley: Unlike overt mechanical failure of an implant, failures in AI systems may not be as obvious. These tools can generate outputs that appear plausible even when they are incorrect. A clinician may only recognize a problem when the output conflicts with their own judgment or with subsequent clinical findings. From a regulatory perspective, agencies currently rely on self-reporting, mainly by manufacturers. In the US, Medical Device Reporting depends on manufacturers reporting adverse events and malfunctions as they become aware of them, and the MDR vigilance system in Europe is similarly led by manufacturers. While important, these systems are passive by design and not well suited to detecting gradual performance drift or systematic bias in AI systems. The EU AI Act strengthens expectations around postmarket monitoring in the EU by requiring manufacturers of high-risk AI systems such as medical devices to implement active postmarket monitoring, logging, and risk management processes. However, it doesn’t replace existing vigilance systems or create an independent, real-time surveillance infrastructure. Academic medical professionals will likely play a large role in the assessment of new devices. Prospective cohort studies or observational data will form a cornerstone to the policing of AI systems. Establishment of registries, in conjunction with regulatory bodies, should be driven by medical and surgical associations such as the American Academy of Orthopaedic Surgeons (AAOS).
Paul Manner (Mon,) studied this question.