RELATE: A Schema-Agnostic Perceiver Encoder for Multimodal Relational Graphs

TL;DR

RELATE tackles the challenge of schema-dependent encoders for relational multi-table data modeled as heterogeneous temporal graphs. It introduces a schema-agnostic encoder that uses shared modality-specific encoders for numerical, timestamp, categorical, and textual data, combined with a Perceiver-style cross-attention bottleneck over a small set of latent tokens to produce fixed-size node representations. The approach achieves competitive performance (within about 3% of schema-specific encoders) on RelBench while substantially reducing parameters (up to 5× fewer) and remains plug-and-play with standard GNN backbones, supporting multi-dataset pretraining and foundation-model-scale learning for relational graphs. This work advances scalable, generalizable learning across diverse schemas and modalities, with practical impact for real-world relational databases and multi-task graph analytics.

Abstract

Relational multi-table data is common in domains such as e-commerce, healthcare, and scientific research, and can be naturally represented as heterogeneous temporal graphs with multi-modal node attributes. Existing graph neural networks (GNNs) rely on schema-specific feature encoders, requiring separate modules for each node type and feature column, which hinders scalability and parameter sharing. We introduce RELATE (Relational Encoder for Latent Aggregation of Typed Entities), a schema-agnostic, plug-and-play feature encoder that can be used with any general purpose GNN. RELATE employs shared modality-specific encoders for categorical, numerical, textual, and temporal attributes, followed by a Perceiver-style cross-attention module that aggregates features into a fixed-size, permutation-invariant node representation. We evaluate RELATE on ReLGNN and HGT in the RelBench benchmark, where it achieves performance within 3% of schema-specific encoders while reducing parameter counts by up to 5x. This design supports varying schemas and enables multi-dataset pretraining for general-purpose GNNs, paving the way toward foundation models for relational graph data.

Figures (3)

• Figure 1: Overview of the RELATE architecture. Columns are encoded by shared modality-specific encoders, aggregated with column metadata, and summarized by a cross-attention transformer into fixed-size node embeddings for GNNs. Please refer to Appendix \ref{['app:col_agg']} for details on how column metadata is aggregated with cell embeddings.
• Figure 2: (A) Parameter comparison across RelBench datasets. We compare the number of trainable parameters between schema-specific encoders (Std) and RELATE. Schema-specific encoders grow with the number of tables and features, whereas RELATE maintains a nearly constant footprint. Percentages above the RELATE bars indicate the parameter ratio relative to schema-specific encoders. (B) Performance v/s parameter count (log scale) for the HGT backbone. Each line connects a schema-specific encoder (blue) to its RELATE counterpart (green), illustrating the change in both performance and model size. RELATE achieves comparable AUC while using up to 5× fewer parameters.
• Figure 3: Ablation: Attention ablation for RelGNN, reporting average AUC performance relative to cross-attention. Bars show the performance of self-attention compared to cross-attention within each dataset.