Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

No Need to Train Your RDB Foundation Model

Linjie Xu, Yanlin Zhang, Quan Gan, Minjie Wang, David Wipf

TL;DR

The paper tackles scalable predictive modeling on relational databases using in-context learning without retraining. It introduces JUICE, a parameter-free encoder that compresses RDB neighborhoods vertically within individual columns to fixed-length representations, preserving column identities so they remain compatible with single-table ICL heads. The authors prove that such encoders need not be trainable and demonstrate a practical pathway (RDBLearn) to deploy these ideas with SQL-backed implementations and zero-training predictions, achieving competitive performance on RelBench and 4DBInfer while offering strong efficiency. The work provides analytical and empirical support for preferring JUICE over dense cross-column embeddings in ICL regimes and presents a compelling open-source toolkit for real-world use. Overall, this advances zero-shot, cross-database predictive modeling by bridging multi-table RDBs with established single-table ICL decoders.

Abstract

Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes. But since the space of potential targets is vast across enterprise settings, how can we \textit{avoid retraining} a new model each time we wish to predict a new quantity of interest? Foundation models based on in-context learning (ICL) offer a convenient option, but so far are largely restricted to single-table operability. In generalizing to multiple interrelated tables, it is essential to compress variably-sized RDB neighborhoods into fixed-length ICL samples for consumption by the decoder. However, the details here are critical: unlike existing supervised learning RDB pipelines, we provide theoretical and empirical evidence that ICL-specific compression should be constrained \emph{within} high-dimensional RDB columns where all entities share units and roles, not \textit{across} columns where the relevance of heterogeneous data types cannot possibly be determined without label information. Conditioned on this restriction, we then demonstrate that encoder expressiveness is actually not compromised by excluding trainable parameters. Hence we arrive at a principled family of RDB encoders that can be seamlessly paired with already-existing single-table ICL foundation models, whereby no training or fine-tuning is required. From a practical standpoint, we develop scalable SQL primitives to implement the encoder stage, resulting in an easy-to-use open-source RDB foundation model\footnote{\label{foot: RDBLearn_learn} https://github.com/HKUSHXLab/rdblearn} capable of robust performance on unseen datasets out of the box.

No Need to Train Your RDB Foundation Model

TL;DR

The paper tackles scalable predictive modeling on relational databases using in-context learning without retraining. It introduces JUICE, a parameter-free encoder that compresses RDB neighborhoods vertically within individual columns to fixed-length representations, preserving column identities so they remain compatible with single-table ICL heads. The authors prove that such encoders need not be trainable and demonstrate a practical pathway (RDBLearn) to deploy these ideas with SQL-backed implementations and zero-training predictions, achieving competitive performance on RelBench and 4DBInfer while offering strong efficiency. The work provides analytical and empirical support for preferring JUICE over dense cross-column embeddings in ICL regimes and presents a compelling open-source toolkit for real-world use. Overall, this advances zero-shot, cross-database predictive modeling by bridging multi-table RDBs with established single-table ICL decoders.

Abstract

Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes. But since the space of potential targets is vast across enterprise settings, how can we \textit{avoid retraining} a new model each time we wish to predict a new quantity of interest? Foundation models based on in-context learning (ICL) offer a convenient option, but so far are largely restricted to single-table operability. In generalizing to multiple interrelated tables, it is essential to compress variably-sized RDB neighborhoods into fixed-length ICL samples for consumption by the decoder. However, the details here are critical: unlike existing supervised learning RDB pipelines, we provide theoretical and empirical evidence that ICL-specific compression should be constrained \emph{within} high-dimensional RDB columns where all entities share units and roles, not \textit{across} columns where the relevance of heterogeneous data types cannot possibly be determined without label information. Conditioned on this restriction, we then demonstrate that encoder expressiveness is actually not compromised by excluding trainable parameters. Hence we arrive at a principled family of RDB encoders that can be seamlessly paired with already-existing single-table ICL foundation models, whereby no training or fine-tuning is required. From a practical standpoint, we develop scalable SQL primitives to implement the encoder stage, resulting in an easy-to-use open-source RDB foundation model\footnote{\label{foot: RDBLearn_learn} https://github.com/HKUSHXLab/rdblearn} capable of robust performance on unseen datasets out of the box.
Paper Structure (50 sections, 6 theorems, 19 equations, 10 figures)

This paper contains 50 sections, 6 theorems, 19 equations, 10 figures.

Table of Contents

  1. Introduction
  2. RDB Predictive Modeling
  3. A Generic RDB Supervised Learning Pipeline
  4. RDB Foundation Models
  5. Schema-agnostic models.
  6. ICL-based models.
  7. Limitations
  8. A Foundation for RDB Foundation Models
  9. An RDB Encoder Specifically for ICL
  10. A 1D GNN Implementation
  11. 1D column-wise subgraph formation.
  12. Meta-path GNN layers.
  13. Resulting JUICE encoder design.
  14. Additional JUICE variations.
  15. Encoder Training is Not Necessary
  16. ...and 35 more sections

Key Result

Proposition 3.2

If $\sigma$ and $\operatorname{agg}$ are positively homogeneous functions, we initialize embeddings using (eq:1D_subgraph), and $\{w_0^{(h)}, w^{(h)} \} \in \mathbb{R}_+$$\forall h$, then without loss of generality (eq:MP-GNN_layer) can be reparameterized with no internal weights.

Figures (10)

  • Figure 1: Example RDB (w/ $K=3$ tables) where task-dependent column importance cannot be determined at the encoder stage. ICL samples are needed to resolve the intrinsic column ambiguity.
  • Figure 2: Entity classification results (AUC) on RelBench; yellow is best, orange is second best among untrained models.
  • Figure 3: RDBLearn ablation (mean AUC on RelBench).
  • Figure 4: Entity regression results (MAE) on RelBench.
  • Figure 5: Entity classification ranking on 4DBInfer.
  • ...and 5 more figures

Theorems & Definitions (8)

  • Definition 3.1
  • Proposition 3.2
  • Definition 4.1
  • Proposition 4.2
  • Proposition 4.3
  • Proposition 6.1
  • Proposition 6.2
  • Proposition 6.3