What happens when ‘AI kills a patient’?

What happens when problems with an Artificial Intelligence (AI) system are implicated in the harm, or death, of one or more patients? The news headlines almost write themselves: "AI kills patient". While the headline may be hyperbole, the scenario is not. AI technologies are being rapidly deployed across healthcare and, like all new technologies, will bring new forms of risk and the potential for unwelcome surprise. Realising the benefits of AI in healthcare will involve carefully managing these new risks and uncertainties. This paper explores four key sets of interconnected questions. First, how will healthcare systems reliably identify and detect when AI technologies are implicated in safety events in the first place? Second, how will stakeholders collaborate, disclose and share relevant information? Third, how (and by whom) will the events, technologies and underlying risks be investigated, analysed and explained? And fourth, how will accountability be negotiated and improvement be coordinated to ensure system-wide learning? Healthcare systems will need to build capabilities that support systemic investigation, expert analysis and collaborative learning around emerging AI risks. Otherwise, the promising potential of AI to improve healthcare may be unnecessarily curtailed when the first AI safety events that harm a patient begin to hit the headlines.