TabR1: Taming GRPO for tabular reasoning LLMs

TL;DR

TabR1 tackles the challenge of enabling reliable, interpretable tabular reasoning with LLMs by introducing Permutation Relative Policy Optimization (PRPO) and a tabular serialization procedure. TabR1 serializes tabular data into natural-language prompts and applies PRPO to encode column-permutation invariance as a structural prior, turning sparse outcome rewards into denser learning signals via intra- and inter-permutation advantages. On 139 OpenML datasets, TabR1 achieves strong fully supervised performance, while also delivering competitive zero-shot and few-shot generalization, with the 8B variant outperforming much larger LLMs in several tasks and approaching 32-shot baselines in zero-shot settings. Overall, PRPO enables efficient, interpretable tabular reasoning in reasonably sized LLMs, offering practical transferability and transparency for real-world tabular prediction tasks.

Abstract

Tabular prediction has traditionally relied on gradient-boosted decision trees and specialized deep learning models, which excel within tasks but provide limited interpretability and weak transfer across tables. Reasoning large language models (LLMs) promise cross-task adaptability with trans- parent reasoning traces, yet their potential has not been fully realized for tabular data. This paper presents TabR1, the first reasoning LLM for tabular prediction with multi-step reasoning. At its core is Permutation Relative Policy Optimization (PRPO), a simple yet efficient reinforcement learning method that encodes column-permutation invariance as a structural prior. By construct- ing multiple label-preserving permutations per sample and estimating advantages both within and across permutations, PRPO transforms sparse rewards into dense learning signals and improves generalization. With limited supervision, PRPO activates the reasoning ability of LLMs for tabular prediction, enhancing few-shot and zero-shot performance as well as interpretability. Comprehensive experiments demonstrate that TabR1 achieves performance comparable to strong baselines under full-supervision fine-tuning. In the zero-shot setting, TabR1 approaches the performance of strong baselines under the 32-shot setting. Moreover, TabR1 (8B) substantially outperforms much larger LLMs across various tasks, achieving up to 53.17% improvement over DeepSeek-R1 (685B).

Figures (10)

• Figure 1: (a) We collect 139 datasets to construct a PRPO-compatible reinforcement learning dataset. Each training sample is permuted into multiple variants, paired with a prediction question, and fed into TabR1. The rollouts generate both intra-permutation and inter-permutation rewards, providing effective optimization signals for TabR1. (b) Win–Tie–Loss comparison between TabR1 and other models under fully trained, few-shot, and zero-shot settings. (c) Cross-domain performance of TabR1 and seven LLMs across seven distinct domains.
• Figure 2: Overview of PRPO. Given a tabular sample, PRPO first generates multiple column-permuted variants that preserve the original label. Each permuted sample is then serialized, paired with the task question $Q$, and passed to TabR1 for rollout to produce candidate reasoning outputs $\{o_{i,j}\}$. Rewards are computed via rule-based evaluation against verifiable ground-truth labels. Next, intra-permutation advantages $\{A_{i,j}^{(1)}\}$ are estimated within each permutation group, while inter-permutation advantages $\{A_{i,j}^{(2)}\}$ are aggregated across permutations. Finally, the two levels of advantages are integrated into $\{A_{i,j}\}$, where $A_{i,j} = \alpha A_{i,j}^{(1)} + \beta A_{i,j}^{(2)}$, enabling permutation-aware reinforcement learning fine-tuning.
• Figure 3: Performance comparison between PRPO and GRPO during fine-tuning on classification and regression datasets under the fully trained setting.
• Figure 4: Domain distribution of fully trained and zero-/few-shot datasets.
• Figure 5: Example of the text template used for a classification task in TabR1.