Subject Information Extraction for Novelty Detection with Domain Shifts

TL;DR

The paper addresses unsupervised novelty detection (UND) in the presence of domain shifts, where normal test samples may come from unseen backgrounds. It introduces Subject-Novelty Detection (SND), which disentangles subject information from background variation by minimizing the mutual information $I(\boldsymbol{z}_s;\boldsymbol{z}_b)$ between subject and background representations and by modeling background with a deep Gaussian Mixture Model over $\boldsymbol{z}_b$. Novelty detection is performed on subject representations $\boldsymbol{z}_s$ via Kernel Density Estimation (KDE), using a KDE-based novelty score $NS=-\hat{p}(\boldsymbol{z}_s)$. The method optimizes a total loss $L_{total}=L_{rec}+\omega_1 E(\boldsymbol{z}_b)+\omega_2 \hat{I}_{MI}(\boldsymbol{z}_s;\boldsymbol{z}_b)$ and demonstrates state-of-the-art performance on Multi-background MNIST, Multi-background Fashion-MNIST, and Kurcuma under substantial domain shifts. The results suggest strong practical impact for real-world UND tasks where background conditions vary, such as medical, security, and robotics settings.

Abstract

Unsupervised novelty detection (UND), aimed at identifying novel samples, is essential in fields like medical diagnosis, cybersecurity, and industrial quality control. Most existing UND methods assume that the training data and testing normal data originate from the same domain and only consider the distribution variation between training data and testing data. However, in real scenarios, it is common for normal testing and training data to originate from different domains, a challenge known as domain shift. The discrepancies between training and testing data often lead to incorrect classification of normal data as novel by existing methods. A typical situation is that testing normal data and training data describe the same subject, yet they differ in the background conditions. To address this problem, we introduce a novel method that separates subject information from background variation encapsulating the domain information to enhance detection performance under domain shifts. The proposed method minimizes the mutual information between the representations of the subject and background while modelling the background variation using a deep Gaussian mixture model, where the novelty detection is conducted on the subject representations solely and hence is not affected by the variation of domains. Extensive experiments demonstrate that our model generalizes effectively to unseen domains and significantly outperforms baseline methods, especially under substantial domain shifts between training and testing data.

Figures (6)

• Figure 1: An illustration of the novelty detection task. The training data (left) consists of images of the digit '0' presented in four backgrounds. The testing data (right) includes images of multiple digits (0-9) in seen backgrounds and entirely unseen backgrounds. Although the '0' digits in the test set are normal, some of them are likely to be labelled as novel due to the shift in background.
• Figure 2: An overview of the proposed SND model.
• Figure 3: t-SNE visualizations illustrating the separation of features: (a) Subject vs. Background, (b) Subject vs. Original, and (c) Background vs. Original. We choose the class of “0” in Multi-background MNIST to provide the visualization result.
• Figure 4: AUROC performance comparison across different image features (Subject, Background, and Original) for novelty detection.
• Figure 5: Visualization of the Multi-background MNIST dataset(a) and Multi-background Fashion-MNIST dataset(b).