From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection

TL;DR

OutFormer tackles the challenge of deploying outlier detection on new tabular tasks in the absence of labeled outliers by leveraging zero-shot in-context learning with a foundation-model approach. It extends FoMo-0D with a diverse mixture of synthetic priors (Gaussian Mixtures, Structural Causal Models, and Copulas) and a self-evolving curriculum guided by a multi-armed bandit to maximize learning progress across heterogeneous tasks. The model is pretrained entirely on synthetic data and, at inference, labels test instances through a forward pass using training data as context, enabling plug-and-play deployment with minimal latency. Empirical results across AdBench and two newly released large-scale OD benchmarks (OddBench and OvRBench) show state-of-the-art performance and robust generalization, with ablations validating the importance of priors diversity, the SEC curriculum, and context-based ensembling. This work highlights the practical potential of tabular foundation models for zero-shot OD and lays groundwork for future improvements in priors, training dynamics, and context optimization.

Abstract

Outlier detection (OD) is widely used in practice; but its effective deployment on new tasks is hindered by lack of labeled outliers, which makes algorithm and hyperparameter selection notoriously hard. Foundation models (FMs) have transformed ML, and OD is no exception: Shen et. al. (2025) introduced FoMo-0D, the first FM for OD, achieving remarkable performance against numerous baselines. This work introduces OUTFORMER, which advances FoMo-0D with (1) a mixture of synthetic priors and (2) self-evolving curriculum training. OUTFORMER is pretrained solely on synthetic labeled datasets and infers test labels of a new task by using its training data as in-context input. Inference is fast and zero-shot, requiring merely forward pass and no labeled outliers. Thanks to in-context learning, it requires zero additional work-no OD model training or bespoke model selection-enabling truly plug-and-play deployment. OUTFORMER achieves state-of-the-art performance on the prominent AdBench, as well as two new large-scale OD benchmarks that we introduce, comprising over 1,500 datasets, while maintaining speedy inference.

Figures (20)

• Figure 1: OutFormer advances FMs for tabular OD and outperforms FoMo-0Dshen2025fomod and other baselines on ADBench.Best and second-best are highlighted. Reported are five relative performance metrics across baselines, and Win/Lose/Tie is the fraction of datasets where OutFormer wins/loses/ties against baseline, while $p$-values $\leq$$0.05$ of the paired permutation test between baseline and OutFormer indicate OutFormer's performance is significantly better. See Appx. Table \ref{['tab:adbench_results_aupr']} for similar results w.r.t. AUPRC, and Appx. Tables \ref{['tab:appendix_fulladbench_aucroc']} (AUROC) and \ref{['tab:appendix_fulladbench_aucpr']} (AUPRC) for performances on individual datasets for all methods.
• Figure 2: Comparison of different curriculum strategies. OutFormer trained using SEC(Ours) strategy outperforms its variants trained with alternative curricula. Metrics are evaluated on ADBench (real-world), and the last column reports overall AUROC on SynBench (in-distribution). Best results are highlighted.
• Figure 3: Proposed OutFormer framework for tabular outlier detection. Pretraining capitalizes on diverse labeled synthetic datasets from a mixture of data priors, and employs self-evolving curriculum training based on multi-armed bandits to estimate the masked test labels. At inference, (frozen) OutFormer estimates the test labels of a real-world task via forward pass only while ingesting training data as context. Speedy inference enables ensembling OutFormer over different subsampled examples and dimensions in the context.
• Figure 4: Our proposed priors are complementary and non-trivial. We train the original FoMo-0D model from shen2025fomod, while replacing its GMM prior with alternative priors and report avg. AUROC performance across synthetic datasets from individual priors. Best and Second are highlighted. The prominent diagonal shows that models trained on individual priors generalize better within than across prior distributions. Even well-established baselines --kNN (averaged over $k \in \{5, 10, 20, 50\}$) and Isolation Forest (IForest) with default hyperparameters -- do not readily achieve top performance.
• Figure 5: Our proposed self-evolving curriculum (SEC) empowers mixed-prior training. Naïve mixed-prior training underperforms GMM-only training on GMM datasets and ADBench. Training on mixed-priors using SEC boosts performance on all priors as well as ADBench as compared to no curriculum.