A Multi-Modal AI System for Screening Mammography: Integrating 2D and 3D Imaging to Improve Breast Cancer Detection in a Prospective Clinical Study

TL;DR

The study addresses persistent false-positive recalls in mammography screening by proposing a multi-modal AI system that jointly leverages FFDM, synthetic 2D (C-View), and DBT imaging. Built on a YOLOX-based architecture, it produces both bounding-box localizations and breast-level predictions, optimized via a dual-training objective and a cross-modality ensemble that preserves depth information from DBT. The system demonstrates strong standalone performance (internal AUROC up to $0.953$) and solid external generalizability across diverse datasets, with an improved version further narrowing the gap to perfect performance. In a prospective deployment across 18 sites, AI support altered recall decisions, reducing recalls for low-risk (green) cases among experienced radiologists and highlighting a potential path to reducing unnecessary procedures, while also identifying areas where AI increased recalls for gray and mixed cases. Overall, the work shows the practical promise of integrating all available mammography modalities to enhance screening efficiency, though future studies are needed to confirm patient-level outcomes and refine bounding-box deployment in clinical workflows.

Abstract

Although digital breast tomosynthesis (DBT) improves diagnostic performance over full-field digital mammography (FFDM), false-positive recalls remain a concern in breast cancer screening. We developed a multi-modal artificial intelligence system integrating FFDM, synthetic mammography, and DBT to provide breast-level predictions and bounding-box localizations of suspicious findings. Our AI system, trained on approximately 500,000 mammography exams, achieved 0.945 AUROC on an internal test set. It demonstrated capacity to reduce recalls by 31.7% and radiologist workload by 43.8% while maintaining 100% sensitivity, underscoring its potential to improve clinical workflows. External validation confirmed strong generalizability, reducing the gap to a perfect AUROC by 35.31%-69.14% relative to strong baselines. In prospective deployment across 18 sites, the system reduced recall rates for low-risk cases. An improved version, trained on over 750,000 exams with additional labels, further reduced the gap by 18.86%-56.62% across large external datasets. Overall, these results underscore the importance of utilizing all available imaging modalities, demonstrate the potential for clinical impact, and indicate feasibility of further reduction of the test error with increased training set when using large-capacity neural networks.

Figures (13)

• Figure 1: An overview of the AI system. a To build the AI system, we collected screening and diagnostic mammography images that contain FFDM, C-View and DBT images. b For each breast, we determined a cancer label based on the pathology reports for the patient within a timeframe of 0 to 120 days from the study date. c To enhance the diversity of the training data, images underwent data augmentation, including affine transformations and random horizontal flips. d Each neural network model in the proposed AI system is trained to create not only bounding-box predictions but also image-level prediction by aggregating information from the top bounding box predictions. e The AI system analyzes all images for each exam and generates probability predictions at the breast level. An exam is classified as "all green" if both breasts receive predictions indicating a low likelihood of cancer. It is labeled "mixed" when only one breast has a low prediction, while the exam is categorized as "gray" if neither breast is deemed unlikely to have cancer. f We evaluated the system on an internal test set (AUROC: 0.945, 95% CI: 0.930, 0.960, N=38,368 breasts) as well as seven external datasets collected across three continents. g In clinical implementation, radiologists review the input images along with the model's predictions ("all green," "mixed," or "gray") to make informed decisions about whether to recall the patient for additional imaging or follow-up.
• Figure 2: Visualization of the bounding-box predictions of the AI system (V2 model). From left to right, R-CC, L-CC, R-MLO, and L-MLO views are displayed. The bounding-box predictions generated from FFDM, C-View, DBT images are ensembled together and displayed on C-View images. The boxes with the highest malignancy prediction (displayed in the brightest green color) closely match the ground-truth lesions (shown as red boxes). This example contains two spiculated masses at right 9:00, anterior depth and right 9:00, posterior depth which underwent ultrasound core biopsies yielding invasive mammary carcinoma with lobular and ductal features. Additionally, the AI system detected a benign lymph node in the right axilla in the top portion of the R-MLO view.
• Figure 3: Retrospective performance analysis of the V1 model on the V1 test set. Exam-level predictions were generated by taking the maximum of the two breast-level predictions for each exam. Thresholds were applied at various AI score percentiles to evaluate sensitivity, specificity, false-negative rate, false positive rate, and recall savings, assuming that exams below the threshold would not be reviewed by a radiologist. a Sensitivity (blue, left axis) and false-negative rate (orange, right axis) as a function of the AI score percentile threshold. Sensitivity decreases and the false-negative rate increases as the threshold rises. b Specificity (blue, left axis) and false positive rate (orange, right axis) plotted against the AI score percentile threshold. They appear as almost flat lines as the number of exams with cancer is very small compared to the entire test set, leading to minimal variation in these metrics across thresholds. c Fraction of recalls saved (orange) as a function of the AI score percentile threshold. At a 43.8th percentile threshold for AI as a standalone reader, 31.7% of recalls originally made by the radiologist could hypothetically be avoided without missing any breast cancers, illustrating the potential of AI to reduce the recall rates in this retrospective analysis.
• Figure 4: The AI assessment box viewed by radiologists during the interpretation of a screening mammogram. A result is assigned to each breast, which can be either green or gray. A green result indicates that the AI model's output is below the operating point threshold, while a gray result signifies that the AI findings are "noncontributory".
• Figure 5: Comparison of AIR by Reader for all cases. 8 of the 20 interpreting radiologists (3-28 years of experience) demonstrated recall rates >12% in the 8 month study period prior to clinical implementation of the AI model. After the introduction of the AI model, 2 of these 8 radiologists (6 and 21 years of experience; interpreted 429 and 1491 exams prior to AI implementation and 645 and 1212 exams after AI implementation, respectively) demonstrated a significant reduction in recall rate (p = 0.02406, z = -2.256, Cohen's h = -0.139 and p = 0.01115, z = -2.538, Cohen's h = -0.099 respectively). 3 of 20 radiologists demonstrated a significant increase (p = 0.01750, z = 2.376, Cohen's h = 0.088; p < 0.00001, z = 5.016, Cohen's h = 0.083; and p < 0.00001, z = 7.631, Cohen's h = 0.148 respectively) in recall rate (3, 8, 28 years of experience; interpreted 1574, 7131, and 5990 prior to AI implementation and 1363, 7471, and 4641 exams after AI implementation, respectively). The remaining 15 of 20 radiologists demonstrated no significant change to their recall rates after the AI clinical implementation (Supplementary Table \ref{['tab:my_table']}). * signifies statistically significant decrease in AIR after AI implementation. signifies statistically significant increase in AIR after AI implementation. Gray shading signifies the ideal AIR 5-12% according to national benchmarks.