Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA

Fengyu Li, Junhao Zhu, Kaishi Song, Lu Chen, Zhongming Yao, Tianyi Li, Christian S. Jensen

TL;DR

Operation-R1 is proposed, the first framework that trains lightweight LLMs via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step and develops two complementary mechanisms: operation merge, which filters spurious operations through multi-candidate consensus, and adaptive rollback, which offers runtime protection against information loss in data transformation.

Abstract

Table Question Answering (TQA) aims to answer natural language questions over structured tables. Large Language Models (LLMs) enable promising solutions to this problem, with operator-centric solutions that generate table manipulation pipelines in a multi-step manner offering state-of-the-art performance. However, these solutions rely on multiple LLM calls, resulting in prohibitive latencies and computational costs. We propose Operation-R1, the first framework that trains lightweight LLMs (e.g., Qwen-4B/1.7B) via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step. To train such an LLM, we first introduce a self-supervised rewarding mechanism to automatically obtain fine-grained pipeline-wise supervision signals for LLM training. We also propose variance-aware group resampling to mitigate training instability. To further enhance robustness of pipeline generation, we develop two complementary mechanisms: operation merge, which filters spurious operations through multi-candidate consensus, and adaptive rollback, which offers runtime protection against information loss in data transformation. Experiments on two benchmark datasets show that, with the same LLM backbone, Operation-R1 achieves average absolute accuracy gains of 9.55 and 6.08 percentage points over multi-step preparation baselines, with 79\% table compression and a 2.2$\times$ reduction in monetary cost.

Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA

TL;DR

Operation-R1 is proposed, the first framework that trains lightweight LLMs via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step and develops two complementary mechanisms: operation merge, which filters spurious operations through multi-candidate consensus, and adaptive rollback, which offers runtime protection against information loss in data transformation.

Abstract

Table Question Answering (TQA) aims to answer natural language questions over structured tables. Large Language Models (LLMs) enable promising solutions to this problem, with operator-centric solutions that generate table manipulation pipelines in a multi-step manner offering state-of-the-art performance. However, these solutions rely on multiple LLM calls, resulting in prohibitive latencies and computational costs. We propose Operation-R1, the first framework that trains lightweight LLMs (e.g., Qwen-4B/1.7B) via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step. To train such an LLM, we first introduce a self-supervised rewarding mechanism to automatically obtain fine-grained pipeline-wise supervision signals for LLM training. We also propose variance-aware group resampling to mitigate training instability. To further enhance robustness of pipeline generation, we develop two complementary mechanisms: operation merge, which filters spurious operations through multi-candidate consensus, and adaptive rollback, which offers runtime protection against information loss in data transformation. Experiments on two benchmark datasets show that, with the same LLM backbone, Operation-R1 achieves average absolute accuracy gains of 9.55 and 6.08 percentage points over multi-step preparation baselines, with 79\% table compression and a 2.2 reduction in monetary cost.
Paper Structure (22 sections, 13 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 22 sections, 13 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. PRELIMINARIES
  3. Task Formulation
  4. RL with Verifiable Rewards
  5. Operation-R1
  6. Framework Overview
  7. ORPO: Operation-wise Group Relative Policy Optimization
  8. Self-supervised Pipeline Reward Mechanism
  9. LLM Post-training by ORPO
  10. Operation Merge
  11. Adaptive Rollback
  12. EXPERIMENTS
  13. Experiments Setup
  14. Overall Performance
  15. Efficiency Analysis
  16. ...and 7 more sections

Figures (6)

  • Figure 1: (a) Direct text-based reasoning, (b) Program-based reasoning, (c) Operation-based multi-agent framwork, and (d) Pipeline-specific model.
  • Figure 2: Overview of Operation-R1, including ORPO-based offline training, parallel operator generation, operation-tree merging, and adaptive rollback for reliable tabular QA.
  • Figure 3: Efficiency-Accuracy-Cost Trade-off Comparison
  • Figure 4: Training curves of Operation-R1-1.7B with and without VGR strategy. Removing VGR causes a pronounced early drop in reward and slower subsequent improvement, whereas the full VGR strategy yields smoother, steadily increasing curves and higher final rewards.
  • Figure 5: Impact of Operator Sequence Length on WikiTQ and TabFact.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Definition 1: Cell-focused QA Task