TACTIC for Navigating the Unknown: Tabular Anomaly deteCTion via In-Context inference

Abstract

Anomaly detection for tabular data has been a long-standing unsupervised learning problem that remains a major challenge for current deep learning models. Recently, in-context learning has emerged as a new paradigm that has shifted efforts from task-specific optimization to large-scale pretraining aimed at creating foundation models that generalize across diverse datasets. Although in-context models, such as TabPFN, perform well in supervised problems, their learned classification-based priors may not readily extend to anomaly detection. In this paper, we study in-context models for anomaly detection and show that the unsupervised extensions to TabPFN exhibit unstable behavior, particularly in noisy or contaminated contexts, in addition to the high computational cost. We address these challenges and introduce TACTIC, an in-context anomaly detection approach based on pretraining with anomaly-centric synthetic priors, which provides fast and data-dependent reasoning about anomalies while avoiding dataset-specific tuning. In contrast to typical score-based approaches, which produce uncalibrated anomaly scores that require post-processing (e.g. threshold selection or ranking heuristics), the proposed model is trained as a discriminative predictor, enabling unambiguous anomaly decisions in a single forward pass. Through experiments on real-world datasets, we examine the performance of TACTIC in clean and noisy contexts with varying anomaly rates and different anomaly types, as well as the impact of prior choices on detection quality. Our experiments clearly show that specialized anomaly-centric in-context models such as TACTIC are highly competitive compared to other task-specific methods.

Figures (8)

• Figure 1: Average AUCROC performance (vertical axis) versus computation time (horizontal axis), calculated over multiple datasets, for classical (circles), deep-learning (triangles), and in-context methods (stars). Note that the horizontal axis corresponds to inference time for out-of-the-box models and to the combined fitting and inference time for models that require per-dataset training. In terms of evaluation time, TACTIC falls within the group of fast shallow methods while achieving top performance for both clean and noisy context settings.
• Figure 2: Schematic overview of our approach. By making use of contextual information about data distribution, TACTIC separates query points into nominal and anomalous.
• Figure 3: Anomaly-centric priors used for pretraining TACTIC (nominal data is marked in blue; anomalies marked in red). Using a mixture of classification-based and GMM-based priors with various anomaly types improves the generalization ability and robustness of our method.
• Figure 4: Performance of our model (TACTIC) against classical, deep, and in-context baselines (the unsupervised extension of TabPFN2.5, termed uTabPFN) in the clean-context setting, computed on multiple real-world datasets. The numeric values under the distribution plots indicate the average rank of each method.
• Figure 5: Performance of our model (TACTIC) against classical, deep, and in-context baselines in the contaminated (noisy) context setting, computed on multiple real-world datasets. The numeric values under the distribution plots indicate the average rank of each method.