FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models

Abstract

Tabular foundation models, specifically Prior-Data Fitted Networks (PFNs), have revolutionized outlier detection (OD) by enabling unsupervised zero-shot adaptation to new datasets without training. However, despite their predictive power, these models typically function as opaque black boxes, outputting scalar outlier scores that lack the operational context required for safety-critical decision-making. Existing post-hoc explanation methods are often computationally prohibitive for real-time deployment or fail to capture the epistemic uncertainty inherent in zero-shot inference. In this work, we introduce FoMo-X, a modular framework that equips OD foundation models with intrinsic, lightweight diagnostic capabilities. We leverage the insight that the frozen embeddings of a pretrained PFN backbone already encode rich, context-conditioned relational information. FoMo-X attaches auxiliary diagnostic heads to these embeddings, trained offline using the same generative simulator prior as the backbone. This allows us to distill computationally expensive properties, such as Monte Carlo dropout based epistemic uncertainty, into a deterministic, single-pass inference. We instantiate FoMo-X with two novel heads: a Severity Head that discretizes deviations into interpretable risk tiers, and an Uncertainty Head that provides calibrated confidence measures. Extensive evaluation on synthetic and real-world benchmarks (ADBench) demonstrates that FoMo-X recovers ground-truth diagnostic signals with high fidelity and negligible inference overhead. By bridging the gap between foundation model performance and operational explainability, FoMo-X offers a scalable path toward trustworthy, zero-shot outlier detection.

Figures (9)

• Figure 1: Architecture of FoMo-X.
• Figure 2: Example of a synthetic context dataset used to train FoMo-X. The left plot indicates whether a sample is an outlier and corresponds to the supervision used for the original FoMo-0D head, whereas the remaining plots show the ground-truth targets used to train the auxiliary heads. The middle plot shows outlier severity, and the right plot shows epistemic uncertainty under the FoMo-0D model.
• Figure 3: FoMo-X explanations for the cardio dataset, visualized using PCA pcaAlgo. The red circles highlight prediction errors, corresponding to differences between the ground truth (center left) and the outlier detection head (left). These errors generally occur in regions of low severity (center right) and high uncertainty (right). The blue circle illustrates how severity can help interpret a complex decision boundary.
• Figure 4: Training progress of both proposed heads. While performance continues to improve throughout training, both heads already achieve high accuracy after only a few epochs.
• Figure 5: Predictions of "surely" by the severity head are more reliable than predictions of "likely". This pattern is observed for artificial test data (left) and also generalizes to real-world datasets (right).