Tabular Foundation Models Can Learn Association Rules

TL;DR

This work addresses ARM for tabular data by leveraging Tabular Foundation Models (TFMs) to learn association rules without frequent itemset mining. It introduces a model-agnostic framework for extracting rules from conditional probabilistic models and instantiates it as TabProbe, using TFMs such as TabPFN, TabICL, and TabDPT. Rules are derived via antecedent validation and consequent extraction governed by thresholds $\tau_a$ and $\tau_c$, yielding concise, high-quality rule sets with strong predictive performance, particularly in small data scenarios. The study discusses scalability and limitations (context size, single-target predictions) and outlines directions for extending to additional models, data modalities, and interpretable downstream classifiers.

Abstract

Association Rule Mining (ARM) is a fundamental task for knowledge discovery in tabular data and is widely used in high-stakes decision-making. Classical ARM methods rely on frequent itemset mining, leading to rule explosion and poor scalability, while recent neural approaches mitigate these issues but suffer from degraded performance in low-data regimes. Tabular foundation models (TFMs), pretrained on diverse tabular data with strong in-context generalization, provide a basis for addressing these limitations. We introduce a model-agnostic association rule learning framework that extracts association rules from any conditional probabilistic model over tabular data, enabling us to leverage TFMs. We then introduce TabProbe, an instantiation of our framework that utilizes TFMs as conditional probability estimators to learn association rules out-of-the-box without frequent itemset mining. We evaluate our approach on tabular datasets of varying sizes based on standard ARM rule quality metrics and downstream classification performance. The results show that TFMs consistently produce concise, high-quality association rules with strong predictive performance and remain robust in low-data settings without task-specific training. Source code is available at https://github.com/DiTEC-project/tabprobe.

Figures (8)

• Figure 1: Example association rule learning from tabular foundation models using Algorithm \ref{['alg:tfm-narl']}: (1) split table into features (X) and labels (Y), (2) fit TFM on X and Y, (3) construct probing matrix with marked items (probability 1) and uniform priors for other items, (4) predict per feature, (5) aggregate predictions into reconstruction matrix, (6) apply thresholding to extract association rules.
• Figure 2: Mean execution time on an 8,124-rows dataset (mushroom) as itemset universe size increases. TFMs learn association rules within a minute on modest hardware for small-to-medium tables, whereas only Aerial+ scales to larger sizes (FP-Growth (x): x is min. support threshold).
• Figure 3: Experimental Setting 2 for downstream task evaluation of TFMs and baselines. (1) tabular data is split into training as test folds for cross validation, (2) TFMs and baselines are run on the training fold to learn rules (CBA) or frequent itemsets (CORELS work with frequent itemsets and not rules), (3) learned rules and itemsets are passed to CBA and CORELS to build a rule-based classifier, (4) rule-based classifiers are evaluated on the test fold. This process is repeated for 5-fold across 10 random seeds.
• Figure 4: Classification tasks results with CORELS on small tabular data. TFMs have better or compatible ($\pm 1\%$) predictive performance on 4 out of 5 small datasets, except the Cervical Cancer dataset. This shows that the small number of rules that TFMs learn is indeed more generalizable than the baselines on unseen data.
• Figure 5: Classification task results with CORELS on large tabular data. On large datasets, TFMs have achieved better or comparable performance on all datasets compared to the baselines.

Theorems & Definitions (6)

• Definition 1: Association Rule
• Definition 2: Conditional Probabilistic Model
• Definition 3: Association Validity Function
• Definition 4: Association Rule Learning
• Definition 5: Multi-Target Instantiation
• Definition 6: Single-Target Instantiation