Memorize Early, Then Query: Inlier-Memorization-Guided Active Outlier Detection

TL;DR

The paper addresses unsupervised outlier detection by leveraging the inlier-memorization (IM) effect observed in deep generative models. It introduces IMBoost, a two-phase framework with a warm-up phase that promotes IM and a polarization phase that actively queries informative labels to maximize the gap between inlier and outlier scores, guided by a loss augmented with inlier/outlier terms and dynamically adjusted thresholds. Theoretical analysis proves that IMBoost progressively lowers inlier risk while raising outlier risk, and experiments across 57 ADBench datasets show state-of-the-art performance with substantially lower computational cost. The work demonstrates the practical value of combining IM signals with active learning for robust, scalable outlier detection in diverse domains, and it provides insights into optimal querying strategies and phase-specific hyperparameters.

Abstract

Outlier detection (OD) aims to identify abnormal instances, known as outliers or anomalies, by learning typical patterns of normal data, or inliers. Performing OD under an unsupervised regime-without any information about anomalous instances in the training data-is challenging. A recently observed phenomenon, known as the inlier-memorization (IM) effect, where deep generative models (DGMs) tend to memorize inlier patterns during early training, provides a promising signal for distinguishing outliers. However, existing unsupervised approaches that rely solely on the IM effect still struggle when inliers and outliers are not well-separated or when outliers form dense clusters. To address these limitations, we incorporate active learning to selectively acquire informative labels, and propose IMBoost, a novel framework that explicitly reinforces the IM effect to improve outlier detection. Our method consists of two stages: 1) a warm-up phase that induces and promotes the IM effect, and 2) a polarization phase in which actively queried samples are used to maximize the discrepancy between inlier and outlier scores. In particular, we propose a novel query strategy and tailored loss function in the polarization phase to effectively identify informative samples and fully leverage the limited labeling budget. We provide a theoretical analysis showing that the IMBoost consistently decreases inlier risk while increasing outlier risk throughout training, thereby amplifying their separation. Extensive experiments on diverse benchmark datasets demonstrate that IMBoost not only significantly outperforms state-of-the-art active OD methods but also requires substantially less computational cost.

Figures (9)

• Figure 1: An illustration of IMBoost: (Top)warm-up phase and (Down)polarization phase.
• Figure 2: (1st and 2nd) Trace plots of inlier and outlier risks during the warm-up and polarization phases on the PageBlocks and Thyroid datasets, respectively.
• Figure 3: Averaged test AUC results at the final (5th) round across 57 datasets from ADBench, with standard deviations over three independent runs. All implementations were done by us. Color scheme: red (IMBoost), blue (OC-based), orange (SB-based), and green (DSAD-based).
• Figure 4: Averaged test AUC results (with standard deviations) of IMBoost using different querying strategies: 1) Random (RD), 2) Confidence Poles (CP), and 3) Mixture Model-based decision boundary (MM). Results are reported at the end of each active learning round.
• Figure 5: (1st–5th) AUC results with varying values of hyperparameters: 1) $\lambda_{1,t}$, 2) $\lambda_{2,t}$, 3) $\xi$, 4) $\alpha$, and 5) $T_1$. (6th) Comparison of running time between the IMBoost and other approaches. Each runtime is rescaled relative to that of IMBoost.