Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learning

Lang Cao, Jingxian Xu, Hanbing Liu, Jinyu Wang, Mengyu Zhou, Haoyu Dong, Shi Han, Dongmei Zhang

Abstract

Tables are a fundamental medium for organizing and analyzing data, making table reasoning a critical capability for intelligent systems. Although large language models (LLMs) exhibit strong general reasoning abilities, they still struggle with accurate numerical reasoning over tabular data, particularly in complex table settings beyond simple relational lookup. Spreadsheet formulas provide a powerful and expressive interface for executable symbolic operations, enabling rich reasoning patterns that remain largely underexplored by existing LLMs. In this paper, we introduce Formula-R1, a model trained via Formula Tuning (Fortune), a formula-driven reinforcement learning (RL) framework for table reasoning. Formula Tuning trains LLMs to generate executable spreadsheet formulas for question answering over general tabular data, using execution success and answer correctness as reward signals, thereby reducing reliance on supervised formula annotations. We demonstrate the effectiveness of Formula Tuning through extensive experiments on seven table reasoning benchmarks. It substantially improves LLM performance on table reasoning, particularly for tasks involving complex tables and multi-step numerical computation. Moreover, Formula-R1 consistently outperforms prior methods under controlled comparison settings. Beyond empirical gains, our extensive analyses provide insights into the role of RL in formula-driven table reasoning, highlighting the broader potential of formula-driven RL to enhance reasoning capabilities in LLMs.

Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learning

Abstract

Tables are a fundamental medium for organizing and analyzing data, making table reasoning a critical capability for intelligent systems. Although large language models (LLMs) exhibit strong general reasoning abilities, they still struggle with accurate numerical reasoning over tabular data, particularly in complex table settings beyond simple relational lookup. Spreadsheet formulas provide a powerful and expressive interface for executable symbolic operations, enabling rich reasoning patterns that remain largely underexplored by existing LLMs. In this paper, we introduce Formula-R1, a model trained via Formula Tuning (Fortune), a formula-driven reinforcement learning (RL) framework for table reasoning. Formula Tuning trains LLMs to generate executable spreadsheet formulas for question answering over general tabular data, using execution success and answer correctness as reward signals, thereby reducing reliance on supervised formula annotations. We demonstrate the effectiveness of Formula Tuning through extensive experiments on seven table reasoning benchmarks. It substantially improves LLM performance on table reasoning, particularly for tasks involving complex tables and multi-step numerical computation. Moreover, Formula-R1 consistently outperforms prior methods under controlled comparison settings. Beyond empirical gains, our extensive analyses provide insights into the role of RL in formula-driven table reasoning, highlighting the broader potential of formula-driven RL to enhance reasoning capabilities in LLMs.

Paper Structure

This paper contains 37 sections, 5 theorems, 26 equations, 9 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Task Formulation
  5. Formula Tuning
  6. Experiments
  7. Settings
  8. Performance of Formula-R1 and Formula-R1-Plus Compared to Other Methods
  9. Performance of Formula Learning under SFT and RL
  10. Comparative Analysis of Spreadsheet Formulas, SQL, and Python for Symbolic Table Reasoning
  11. Additional results and further analysis.
  12. Conclusion
  13. Ethical Statement
  14. Theoretical Analysis and Discussion
  15. Textual vs. Symbolic Reasoning in Table Understanding
  16. ...and 22 more sections

Key Result

Theorem 1

Under mild assumptions, the expected reward achieved by symbolic reasoning is greater than or equal to that of textual reasoning for any input $s$:

Figures (9)

  • Figure 1: Overview of Formula Tuning (Fortune).
  • Figure 2: A simplified illustration contrasting Textual versus Symbolic Reasoning in Table Understanding, and Supervised Fine-Tuning (SFT) versus Reinforcement Learning (RL) in Symbolic Table Reasoning.
  • Figure 3: Performance comparison with and without explicit reasoning process under Zero-Shot and Reinforcement Learning (RL) settings across various datasets. Each group of bars shows the accuracy (%) achieved by four configurations: Zero-Shot without Reasoning, Zero-Shot with Reasoning, RL without Reasoning, and RL with Reasoning.
  • Figure 4: Top 10 most frequent formula operators used during evaluation.
  • Figure 5: Prompt for symbolic reasoning with formula. Blue text indicates placeholders for variables within the prompt.
  • ...and 4 more figures

Theorems & Definitions (13)

  • Definition 1: Textual and Symbolic Policies
  • Theorem 1: Symbolic Reasoning Superiority
  • Remark 1: Symbolic Reasoning Potential Benefit
  • Theorem 2: RL Superiority
  • Remark 2: RL Objective and Potential Benefit
  • Lemma 1: Reward Decomposition
  • proof
  • Lemma 2: MLE Minimizes KL Divergence
  • Lemma 3: Convergence of SFT and Reward Upper Bound
  • Remark 3: SFT Bound
  • ...and 3 more