Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I Detect What I Don't Know: Incremental Anomaly Learning with Stochastic Weight Averaging-Gaussian for Oracle-Free Medical Imaging

Nand Kumar Yadav, Rodrigue Rizk, William CW Chen, KC Santosh

TL;DR

The paper tackles unknown anomaly detection in medical imaging under label scarcity by proposing an oracle-free, unsupervised incremental framework that grows the normal data manifold from a small seed. It combines a frozen backbone with lightweight adapters, a PatchCore-based memory (coreset), and SWAG-based epistemic uncertainty to gate admissions via dual distance and uncertainty criteria. The method, IL-PatchCore+SWAG, achieves substantial improvements over baselines on COVID-CXR, Pneumonia CXR, and Brain MRI ND-5, demonstrating high ROC-AUC and PR-AUC while reducing false positives and enhancing localization. This approach offers a scalable, robust solution for real-world medical imaging tasks where labeled anomalies are rare, enabling safe, continual refinement of normality without relying on anomalies or replay buffers.

Abstract

Unknown anomaly detection in medical imaging remains a fundamental challenge due to the scarcity of labeled anomalies and the high cost of expert supervision. We introduce an unsupervised, oracle-free framework that incrementally expands a trusted set of normal samples without any anomaly labels. Starting from a small, verified seed of normal images, our method alternates between lightweight adapter updates and uncertainty-gated sample admission. A frozen pretrained vision backbone is augmented with tiny convolutional adapters, ensuring rapid domain adaptation with negligible computational overhead. Extracted embeddings are stored in a compact coreset enabling efficient k-nearest neighbor anomaly (k-NN) scoring. Safety during incremental expansion is enforced by dual probabilistic gates, a sample is admitted into the normal memory only if its distance to the existing coreset lies within a calibrated z-score threshold, and its SWAG-based epistemic uncertainty remains below a seed-calibrated bound. This mechanism prevents drift and false inclusions without relying on generative reconstruction or replay buffers. Empirically, our system steadily refines the notion of normality as unlabeled data arrive, producing substantial gains over baselines. On COVID-CXR, ROC-AUC improves from 0.9489 to 0.9982 (F1: 0.8048 to 0.9746); on Pneumonia CXR, ROC-AUC rises from 0.6834 to 0.8968; and on Brain MRI ND-5, ROC-AUC increases from 0.6041 to 0.7269 and PR-AUC from 0.7539 to 0.8211. These results highlight the effectiveness and efficiency of the proposed framework for real-world, label-scarce medical imaging applications.

I Detect What I Don't Know: Incremental Anomaly Learning with Stochastic Weight Averaging-Gaussian for Oracle-Free Medical Imaging

TL;DR

The paper tackles unknown anomaly detection in medical imaging under label scarcity by proposing an oracle-free, unsupervised incremental framework that grows the normal data manifold from a small seed. It combines a frozen backbone with lightweight adapters, a PatchCore-based memory (coreset), and SWAG-based epistemic uncertainty to gate admissions via dual distance and uncertainty criteria. The method, IL-PatchCore+SWAG, achieves substantial improvements over baselines on COVID-CXR, Pneumonia CXR, and Brain MRI ND-5, demonstrating high ROC-AUC and PR-AUC while reducing false positives and enhancing localization. This approach offers a scalable, robust solution for real-world medical imaging tasks where labeled anomalies are rare, enabling safe, continual refinement of normality without relying on anomalies or replay buffers.

Abstract

Unknown anomaly detection in medical imaging remains a fundamental challenge due to the scarcity of labeled anomalies and the high cost of expert supervision. We introduce an unsupervised, oracle-free framework that incrementally expands a trusted set of normal samples without any anomaly labels. Starting from a small, verified seed of normal images, our method alternates between lightweight adapter updates and uncertainty-gated sample admission. A frozen pretrained vision backbone is augmented with tiny convolutional adapters, ensuring rapid domain adaptation with negligible computational overhead. Extracted embeddings are stored in a compact coreset enabling efficient k-nearest neighbor anomaly (k-NN) scoring. Safety during incremental expansion is enforced by dual probabilistic gates, a sample is admitted into the normal memory only if its distance to the existing coreset lies within a calibrated z-score threshold, and its SWAG-based epistemic uncertainty remains below a seed-calibrated bound. This mechanism prevents drift and false inclusions without relying on generative reconstruction or replay buffers. Empirically, our system steadily refines the notion of normality as unlabeled data arrive, producing substantial gains over baselines. On COVID-CXR, ROC-AUC improves from 0.9489 to 0.9982 (F1: 0.8048 to 0.9746); on Pneumonia CXR, ROC-AUC rises from 0.6834 to 0.8968; and on Brain MRI ND-5, ROC-AUC increases from 0.6041 to 0.7269 and PR-AUC from 0.7539 to 0.8211. These results highlight the effectiveness and efficiency of the proposed framework for real-world, label-scarce medical imaging applications.

Paper Structure

This paper contains 16 sections, 2 theorems, 4 equations, 7 figures, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Classical Anomaly Detection Methods
  4. Deep Learning-Based Anomaly Detection
  5. Motivation
  6. Proposed Method
  7. Implications.
  8. Incremental Learning with and without an Oracle.
  9. Experimental Setup, Results, and Discussion
  10. Hyperparameter Settings.
  11. Datasets
  12. Results Analysis
  13. Ablation study using SWAG, MC Dropout, and Ensemble
  14. Failed Case Analysis
  15. Limitations
  16. ...and 1 more sections

Key Result

theorem thmcountertheorem

Assume an oracle-assisted variant of Alg. alg:active_rounds, implemented conceptually via strict_normal_only =True. After the dual gate $(z_s,z_u)$ selects candidates satisfying $z_s(x)\!\le\!\tau_z$ and $z_u(x)\!\le\!\tau_z$, the ground-truth label $y(x)$ is queried, and only samples with $y{=}0$ a is identically $\alpha = 0$ for all active-learning rounds, independent of the gate threshold relax

Figures (7)

  • Figure 1: Pipeline of the proposed anomaly-detection framework.
  • Figure 2: Corresponding heatmaps for baseline (left) and proposed IL approach (right) on Chest_xray dataset.
  • Figure 3: PR and ROC curves before and after IL on the Chest_xray dataset.
  • Figure 4: Confusion matrices before and after IL. IL improves performance.
  • Figure 5: Brain MRI dataset representative heatmaps before (left) and after IL (right).
  • ...and 2 more figures

Theorems & Definitions (2)

  • theorem thmcountertheorem: Oracle-assisted zero-contamination under dual-gate prefilter
  • proposition thmcounterproposition: Oracle-free dual-gate bound (image level)