Breast cancer screening is increasingly moving toward a more personalized approach based on individual risk rather than a 1-size-fits-all strategy. Highlighted by the recently reported results of the Women Informed to Screen Depending on Measures of Risk (WISDOM) trial, which randomly assigned US women to risk-based imaging regimens or annual mammography screening, risk-based screening appears feasible and safe.1 More personalized, risk-based strategies could allow women at low risk to have less frequent imaging while those at high risk could receive more intensive screening, including supplemental screening magnetic resonance imaging scans, or risk-reducing medication. The ultimate goal is to achieve earlier breast cancer detection while minimizing potential harms, including unnecessary breast biopsies. Although promising, risk-based screening is not yet widely adopted because current clinical risk prediction tools have limited clinical utility at the individual patient level, despite showing moderate performance in population-based analyses. These models also generally rely on patient-reported clinical information, such as a woman’s demographics and information about her breast history and family history of breast cancer, which can be incomplete or inconsistently captured during routine care. More recently, there has been growing optimism for artificial intelligence (AI)–based breast cancer risk prediction tools based solely on deep learning evaluation of an individual’s mammography images. These AI-based risk prediction tools may streamline risk assessment by relying on a patient’s mammogram images rather than requiring detailed clinical history. To this end, in this issue of the Journal, Kaul et al.2 compare the performance of 3 commonly used clinical breast cancer risk prediction models that require gathering information from patients—Gail; Tyrer-Cuzick, version 8; and Breast Cancer Surveillance Consortium (BCSC), version 3—to MIRAI, an open-source AI image-based model applied to digital screening mammograms obtained from the Mayo Clinic Biobank registry. In their analysis, the authors retrospectively evaluate the performance of 5-year breast cancer risk prediction across multiple models using mammograms performed between 2009 and 2015 from 12 308 women. With robust collection of individual demographic information and clinical risk factors from patient intake forms, mammography results, and linkage to a regional tumor registry for long-term cancer outcomes, the authors determined that MIRAI had similar or better discriminatory accuracy (ie, a model’s ability to separate risk classes from one another) than the clinical models for both overall breast cancer risk (C index = 0.71, 95% CI = 00.68 to 0.74) and invasive breast cancer risk (C index = 00.71, 95% CI = 00.67 to 0.75). Discriminatory accuracy of the 3 clinical risk models was modest, ranging from 0.58 to 0.68 across models, in keeping with other population-level studies, which have shown moderate risk prediction when using these models.3 One unique contribution of this study is that the authors also evaluated calibration (observed to expected ratio) for both overall cancer risk prediction and invasive breast cancer risk prediction. Although discriminatory accuracy provides a measure of a model’s ability to distinguish between individuals who do and do not develop breast cancer, calibration assesses whether the model accurately predicts the proportion of individuals who will develop cancer in a population. For a well-calibrated model, the observed to expected ratio should approximate 1, while ratios greater than 1 indicate risk underestimation and ratios below 1 indicate risk overestimation. Good calibration is especially important in settings where clinical decisions are based on absolute risk estimates, such as recommendations for more intensive screening with magnetic resonance imaging or preventive interventions in high-risk individuals. Kaul et al.2 found that for overall breast cancer risk, MIRAI had better calibration (observed/expected = 0.96, 95% CI = 0.85 to 1.08) than 2 of the clinical risk models: Gail (observed/expected = 1.22, 95% CI = 1.07 to 1.38) and BCSC (observed/expected = 1.38, 95% CI = 1.22 to 1.56). MIRAI demonstrated similar calibration to the Tyrer-Cuzick model combined with volumetric percentage of density and polygenic risk score (observed/expected = 0.99, 95% CI = 0.87 to 1.13). For invasive breast cancer risk, MIRAI tended to overestimate risk (observed/expected = 0.68, 95% CI = 0.58 to 0.78), as did the Tyrer-Cuzick model when combined with volumetric percentage of density (observed/expected = 0.73, 95% CI = 0.62 to 0.85), while the BCSC and Gail models were better calibrated (observed/expected range = 0.86-0.99). These findings are not surprising, given that the BCSC and Gail models were trained and calibrated to predict invasive breast cancer risk specifically, while MIRAI and Tyrer-Cuzick were trained and calibrated to predict overall risk. These findings highlight differences between these models and the importance of choosing a model carefully based on the outcome of greatest interest (any cancer vs invasive cancer only). Although an ultimate goal of risk model estimates may be to help organize population-level screening, their clinical value ultimately depends on how well they inform decisions for individual patients, which is where current models often fall short. Of particular note, Kaul et al.2 found little agreement between which individuals were identified as high risk across all the models (including MIRAI) in their analysis. Indeed, this finding corroborates prior reports demonstrating that different women may be categorized as having high breast cancer risk by different risk models. In a large analysis of 31 115 US women in a multisite health system with screening mammography performed between 2011 and 2015 with long-term cancer follow-up, almost half of women (46.6%) were classified as high risk by at least 1 of the 3 common clinical risk prediction models for the 5-year high-risk threshold of 1.67% or higher (Figure 1).4 Of clinical importance, more than 21% of individuals were classified as high risk by 1 model but average risk by another model. As the Venn diagram in Figure 1 highlights, the individual women identified as high risk by 1 model are often reported as average risk by the other models. This variability in individual-level classification at the patient level is important because recommendations for supplemental screening or preventive therapy may differ depending on which risk model is used. These findings highlight a central challenge for risk-based screening: even models that perform adequately at the population level may not provide consistent or actionable risk estimates for individual patients, underscoring the need for more reliable approaches and evaluations. Agreement of breast cancer risk model results for individual women identified as high risk within 5 years on at least 1 of the breast cancer risk prediction models using clinical data obtained from patients (n = 31 115). The Venn diagram at the bottom presents data for all women identified as high risk by at least 1 model, using shaded, overlapping regions to represent the Boolean operation. The size of each Venn diagram circle corresponds to the number of women identified as high risk by each risk model. Abbreviation: BCSC = Breast Cancer Surveillance Consortium. Adapted from Paige JS, Lee CI, Wang PC, et al. Variability among breast cancer risk classification models when applied at the level of the individual woman. J Gen Intern Med. 2023;38(11):2584-2592. https://doi.org/10.1007/s11606-023-08043-4 In this context, several additional commercial AI image-based risk assessment tools are now entering the market in addition to MIRAI, the focus of this study. These AI tools hold promise, but greater evaluative effort will be required before their widespread adoption. Although the biological or imaging features driving these AI predictions remain incompletely understood, the more immediate issue for clinical implementation is whether these models perform reliably across diverse populations and contemporary screening settings. If AI can provide more consistent and accurate risk measures, it may help strengthen confidence in personalized, risk-based screening approaches. Additional population-level studies using similarly robust data collection measures are needed to examine AI risk model performance in other populations. Moreover, this study was limited to digital mammograms without tomosynthesis because MIRAI cannot currently be applied to examinations with digital breast tomosynthesis. Digital breast tomosynthesis is quickly replacing digital mammography as the most commonly used breast cancer screening modality; therefore, AI risk models will need to be adapted so that they can be used in modern screening settings. Beyond retrospective external validation, prospective, real-world evaluation will be needed before use of AI for breast cancer risk prediction can be recommended.5 Prospective randomized trials such as the WISDOM trial are costly and would take years to complete, however, making such trials impractical for every new commercial tool entering the market. More likely, evidence will need to be generated with a combination of large-scale observational data and pragmatic studies in real-world settings. True learning health systems and observational registries such as the BCSC will be critical for generating this evidence. In summary, Kaul et al.2 provide important evidence for both strengths and caveats of AI tools such as MIRAI for breast cancer risk prediction. More studies are needed to assess AI-driven breast cancer risk prediction in real-world settings and at the level of individual patients. In the meantime, it remains uncertain whether the adoption of AI for risk prediction will translate to meaningful clinical benefits; widespread adoption of AI-driven risk-based screening will be useful only if the eventual health interventions that stem from these risk estimates are proven to be effective. For AI risk prediction tools to improve breast cancer outcomes, clinical guidelines must be updated to incorporate these tools, move toward 5-year risk assessments rather than lifetime risk assessments, and establish when primary prevention and supplemental magnetic resonance imaging screening should be triggered. Christoph I. Lee (Conceptualization; Formal analysis; Writing—original draft; Writing—review Formal analysis; Writing—original draft; Writing—review Formal analysis; Project administration; Writing—original draft; Writing—review & editing) No funding was used for this editorial. No new data were generated or analyzed for this editorial.
Lee et al. (Mon,) studied this question.