Human in control: Shared decision-making with clinical decision-support systems under the Artificial Intelligence Act

Artificial intelligence (AI) is gaining significance in medical decision-making. The AI Act, which entered into force in 2024, regulates not only the development of AI, but also its deployment. In the healthcare context this affects hospitals, clinics, healthcare professionals and their relationship with patients. This article explores to what extent the AI Act brings new tools to regulate the medical shared-decision making process. The AI Act regulates the role of the clinician in decision-making through its human oversight provisions. Art 14 requires providers to make human oversight possible, while Art 26(2) requires deployers to allocate human oversight to someone with the necessary competences. This could constitute a new obligation for deployers. However, human oversight measures would to some extent, have already been implemented previously to comply with Art 22 GDPR or with medical law and medical professional law requiring due care. The AI Act mostly brings the possibility to develop clearer European standards for oversight. The role of patients, as the subjects of automated decision-making, is not directly addressed in the AI Act. The provisions which could have signified a new information requirement for patients (Art 26(11) and Art 86) apply only to some high-risk systems, not to medical devices. Obligations to involve patients therefore stem rather from medical law and, to some extent, GDPR information requirements. The AI Act is unlikely to provide answers to open questions, such as whether health professionals are obliged to inform patients of the use of AI in their care.