Lessons Learned in Building Expertly Annotated Multi-Institution Datasets and Hosting the RSNA AI Challenges

Overview of ChallengesCreating and curating high-quality datasets for medical imaging AI challenges require substantial coordination and collaboration.Obtaining medical imaging data can be difficult because of patient privacy and data security concerns and the resources required to safely extract and This copy is for personal use only.To order printed copies, contact reprints@rsna.orgThe Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017.This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets.The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security.Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources.Overcoming these obstacles requires meticulous project management and adherence to strict timelines.The article also highlights the potential of crowdsourced annotation to progress medical imaging research.Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes.