Selective Test-Time Adaptation for Unsupervised Anomaly Detection using Neural Implicit Representations

TL;DR

This work introduces a novel concept of selective test-time adaptation that utilizes the inherent characteristics of deep pre-trained features to adapt selectively in a zero-shot manner to any test image from an unseen domain.

Abstract

Deep learning models in medical imaging often encounter challenges when adapting to new clinical settings unseen during training. Test-time adaptation offers a promising approach to optimize models for these unseen domains, yet its application in anomaly detection (AD) remains largely unexplored. AD aims to efficiently identify deviations from normative distributions; however, full adaptation, including pathological shifts, may inadvertently learn the anomalies it intends to detect. We introduce a novel concept of selective test-time adaptation that utilizes the inherent characteristics of deep pre-trained features to adapt selectively in a zero-shot manner to any test image from an unseen domain. This approach employs a model-agnostic, lightweight multi-layer perceptron for neural implicit representations, enabling the adaptation of outputs from any reconstruction-based AD method without altering the source-trained model. Rigorous validation in brain AD demonstrated that our strategy substantially enhances detection accuracy for multiple conditions and different target distributions. Specifically, our method improves the detection rates by up to 78% for enlarged ventricles and 24% for edemas.

Figures (4)

• Figure 2: STA-AD at test-time. We leverage source model predictions $\mathbf{x}^{'}_{t}$ as "content" and target images $\mathbf{x}_{t}$ as "style" to train neural implicit representations $\boldsymbol{\phi}_{t}$ to adapt images $\hat{\mathbf{x}}_{t}$. The initial false positives due to domain shifts seen in $\mathbf{x}_{rec}$ are removed after adaptation ($\hat{\mathbf{x}}_{rec}$), while the detected lesion is highlighted.
• Figure 3: Comparison on healthy samples from $T_{+}$ dataset. For both backbones, the source models achieve higher false positives due to inaccurate reconstructions. Our approach, STA-AD, through the adapted image, reduces false positive detections in the presented anomaly maps.
• Figure 4: Comparisons with different backbones on $T_{+}$ unhealthy samples. The STA-AD framework can enhance the performance of AD methods by reducing false positives and accurately detecting anomalies after target adaptation.
• Figure 5: Visual results of histogram matching and our method on the target dataset. These outputs are obtained by using the target input image and source model predictions as inputs. For histogram matching, the outputs result in visually incorrect results, introducing significant artifacts that degrade the final image quality. Whereas our method yields an adapted image that preserves image quality and thereby improves anomaly detection.