Doctor-in-the-Loop: An Explainable, Multi-View Deep Learning Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer

TL;DR

This work tackles the challenge of predicting pathologic response (pR) in NSCLC after neoadjuvant therapy by introducing Doctor-in-the-Loop, a training paradigm that injects expert domain knowledge into an intrinsically explainable deep learning model. It uses a gradual multi-view strategy that starts from global lung regions and progressively focuses on lung and lesion areas, guided by Grad-CAM heatmaps aligned to expert masks via an XAI loss. The approach achieves improved predictive performance and transparent reasoning over ablations and conventional radiomics/deep-feature baselines, demonstrating stronger alignment with clinical reasoning and enhanced trustworthiness. These results suggest a viable path toward clinically integrated, explainable AI for treating NSCLC and potentially other pathologies, especially when data are scarce and interpretability is essential.

Abstract

Non-small cell lung cancer (NSCLC) remains a major global health challenge, with high post-surgical recurrence rates underscoring the need for accurate pathological response predictions to guide personalized treatments. Although artificial intelligence models show promise in this domain, their clinical adoption is limited by the lack of medically grounded guidance during training, often resulting in non-explainable intrinsic predictions. To address this, we propose Doctor-in-the-Loop, a novel framework that integrates expert-driven domain knowledge with explainable artificial intelligence techniques, directing the model toward clinically relevant anatomical regions and improving both interpretability and trustworthiness. Our approach employs a gradual multi-view strategy, progressively refining the model's focus from broad contextual features to finer, lesion-specific details. By incorporating domain insights at every stage, we enhance predictive accuracy while ensuring that the model's decision-making process aligns more closely with clinical reasoning. Evaluated on a dataset of NSCLC patients, Doctor-in-the-Loop delivers promising predictive performance and provides transparent, justifiable outputs, representing a significant step toward clinically explainable artificial intelligence in oncology.

Figures (5)

• Figure 1: Doctor-in-the-Loop training paradigm. Step 1: the model is trained on the global image using only the classification loss $L_{cls}$. Step 2: the segmentation mask $\boldsymbol{M}_{i}$, corresponding to a broad anatomical view, is employed to guide the model's focus using Grad-CAM heatmaps. Both the classification loss $L_{cls}$ and the XAI loss $L_{xai}$ are applied to train the model's weights. Step t+1: the segmentation mask $\boldsymbol{M}_{t}$, corresponding to a detailed anatomical view, is employed to further refine the model's focus through Grad-CAM heatmaps using both the classification loss $L_{cls}$ and the XAI loss $L_{xai}$. The red arrows indicate the forward pass, while the blue arrows represent the backpropagation phase. In our context, represents the Global Image View, represents the Lung View, and represents the Lesion View.
• Figure 2: Grad-CAM heatmaps across the three steps of the Doctor-in-the-Loop approach for six representative patients. Patients are grouped into two rows based on their predicted class: No-pR (absence of pR) and pR (presence of pR). For each patient and each step, three visualizations are shown: the CT images; the corresponding 2D Grad-CAM map of a selected slice (2D map), and the corresponding 3D Grad-CAM map (3D map). Specifically, the global CT image, the CT image with highlighted the lung segmentation, and the CT image with highlighted the lesion segmentation are shown in Step 1, Step 2 and Step 3, respectively. Heatmaps use a color gradient where lower values are shown in blue and higher values in red, reflecting the intensity of the model's focus. The gradual refinement of focus through the steps highlights the progressive enhancement of the model’s explainability. represents the Global Image View, represents the Lung View, and represents the Lesion View.
• Figure 3: Comparison between the Doctor-in-the-Loop (in blue) and XAI-guide (in red) approaches.
• Figure 4: Comparison between the Doctor-in-the-Loop (in blue) and Gradual Learning (in red) approaches.
• Figure 5: Comparison between the Doctor-in-the-Loop (in blue) and Segmentation (in red) approaches.