Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auditing Sybil: Explaining Deep Lung Cancer Risk Prediction Through Generative Interventional Attributions

Bartlomiej Sobieski, Jakub Grzywaczewski, Karol Dobiczek, Mateusz Wójcik, Tomasz Bartczak, Patryk Szatkowski, Przemysław Bombiński, Matthew Tivnan, Przemyslaw Biecek

TL;DR

The paper addresses the need to verify causal reasoning in deep lung cancer risk predictors like Sybil, moving beyond purely observational validation. It introduces S(H)NAP, a model-agnostic auditing framework that uses diffusion-bridge based generative interventions to perturb pulmonary nodules and extract causal attributions, operationalized via SHNAP (nodule removal) and SNAP (nodule insertion) explanations, with gSHNAP extending to region-level causes. The authors demonstrate that Sybil can be approximated as a linear model with main and pairwise nodule interactions (LMPI) and reveal critical misalignments, including radial sensitivity bias and reliance on clinically unjustified artifacts, through rigorous radiologist validation and large-scale perturbation analysis. This work provides a principled, interpretable, and scalable approach to audit high-stakes medical AI systems, with implications for safer clinical deployment and applicability to other 3D medical-imaging models.

Abstract

Lung cancer remains the leading cause of cancer mortality, driving the development of automated screening tools to alleviate radiologist workload. Standing at the frontier of this effort is Sybil, a deep learning model capable of predicting future risk solely from computed tomography (CT) with high precision. However, despite extensive clinical validation, current assessments rely purely on observational metrics. This correlation-based approach overlooks the model's actual reasoning mechanism, necessitating a shift to causal verification to ensure robust decision-making before clinical deployment. We propose S(H)NAP, a model-agnostic auditing framework that constructs generative interventional attributions validated by expert radiologists. By leveraging realistic 3D diffusion bridge modeling to systematically modify anatomical features, our approach isolates object-specific causal contributions to the risk score. Providing the first interventional audit of Sybil, we demonstrate that while the model often exhibits behavior akin to an expert radiologist, differentiating malignant pulmonary nodules from benign ones, it suffers from critical failure modes, including dangerous sensitivity to clinically unjustified artifacts and a distinct radial bias.

Auditing Sybil: Explaining Deep Lung Cancer Risk Prediction Through Generative Interventional Attributions

TL;DR

The paper addresses the need to verify causal reasoning in deep lung cancer risk predictors like Sybil, moving beyond purely observational validation. It introduces S(H)NAP, a model-agnostic auditing framework that uses diffusion-bridge based generative interventions to perturb pulmonary nodules and extract causal attributions, operationalized via SHNAP (nodule removal) and SNAP (nodule insertion) explanations, with gSHNAP extending to region-level causes. The authors demonstrate that Sybil can be approximated as a linear model with main and pairwise nodule interactions (LMPI) and reveal critical misalignments, including radial sensitivity bias and reliance on clinically unjustified artifacts, through rigorous radiologist validation and large-scale perturbation analysis. This work provides a principled, interpretable, and scalable approach to audit high-stakes medical AI systems, with implications for safer clinical deployment and applicability to other 3D medical-imaging models.

Abstract

Lung cancer remains the leading cause of cancer mortality, driving the development of automated screening tools to alleviate radiologist workload. Standing at the frontier of this effort is Sybil, a deep learning model capable of predicting future risk solely from computed tomography (CT) with high precision. However, despite extensive clinical validation, current assessments rely purely on observational metrics. This correlation-based approach overlooks the model's actual reasoning mechanism, necessitating a shift to causal verification to ensure robust decision-making before clinical deployment. We propose S(H)NAP, a model-agnostic auditing framework that constructs generative interventional attributions validated by expert radiologists. By leveraging realistic 3D diffusion bridge modeling to systematically modify anatomical features, our approach isolates object-specific causal contributions to the risk score. Providing the first interventional audit of Sybil, we demonstrate that while the model often exhibits behavior akin to an expert radiologist, differentiating malignant pulmonary nodules from benign ones, it suffers from critical failure modes, including dangerous sensitivity to clinically unjustified artifacts and a distinct radial bias.
Paper Structure (38 sections, 1 theorem, 14 equations, 31 figures, 1 table, 2 algorithms)

This paper contains 38 sections, 1 theorem, 14 equations, 31 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Background
  4. Methodology
  5. Synthetic perturbations
  6. Constructing the explanations
  7. Experiments
  8. Validity of synthetic perturbations
  9. Validity of the explanations
  10. Opening Sybil's black box with SHNAP
  11. Opening Sybil's black box with SNAP
  12. Analyzing the background effect
  13. Discussion and limitations
  14. Extended related works
  15. SHAP and nSV
  16. ...and 23 more sections

Key Result

Theorem 4.1

Let $p(\mathbf{x}^1)$ and $p(\mathbf{x}^2)$ be two probability distributions with time-parameterized evolutions $p(\mathbf{x}_t^1)$ and $p(\mathbf{x}_t^2)$ under the forward process (eq:forward_diffusion). The Kullback-Leibler divergence between them decomposes as: where $\mathbf{D}_\tau = \mathbf{G}_\tau \mathbf{G}_\tau^T$ and $\mathcal{J}$ is the Relative Fisher Information (RFI):

Figures (31)

  • Figure 1: Sybil (bottom) is a frontier model for lung cancer risk prediction from a single CT scan. We propose S(H)NAP (top), a novel framework for auditing such models through diffusion-bridge-based generative interventions on pulmonary nodules (middle). SHNAP (top left) decomposes the model's prediction into individual nodule contributions and inter-nodule interactions by replacing them with healthy tissue. SNAP (top right) probes volumetric sensitivity by systematically inserting nodules of known malignancy, revealing spatial biases in risk estimation.
  • Figure 2: 2D visualizations of our nodule removal (left) and insertion (right) approaches performed on 3D subvolumes of an LDCT scan.
  • Figure 3: Results of an expert study evaluating the realism of nodule removal (a., accuracy; b., response bias) and the preservation of properties during nodule insertion (c.).
  • Figure 4: Accuracy of approximating Sybil as an $\text{LMPI}$ over pulmonary nodules across two datasets, including both first- and second-order effects.
  • Figure 5: Comparison of risk predicted by Sybil and relative nodule contributions across two datasets.
  • ...and 26 more figures

Theorems & Definitions (1)

  • Theorem 4.1: Mismatched estimation, verdu2009mismatched