Who moved my scan? Early adopter experiences with pre- and post-market healthcare AI regulation challenges

Real-world clinical experience provides a much-needed opportunity for deep learning AI algorithms to evolve and improve. Yet, it also constitutes a regulatory challenge, since such potential for learning may essentially change the algorithm and introduce new biases. We focus on the gaps between “lifecycle” regulation and implementation from the perspective of the deployers, addressing three interconnected dimensions: (a) How precautionary regulation affects AI deployment in healthcare, (b) How healthcare providers view explainable AI (XAI), and (c) How AI deployment influences, and is influenced by, team routines in clinical settings. We conclude by suggesting ways in which the ends of healthcare AI regulation and deployment can successfully meet.