Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR

TL;DR

This paper tackles the challenge of reading and reasoning over format-heavy documents by observing that formatted content (e.g., formulas and tables) induces higher output entropy in OCR systems. It introduces Format Decoupled Reinforcement Learning (FD-RL), a two-stage SFT-then-RL framework that leverages entropy-based data filtration and format-specific rewards to optimize format-level validity rather than token-level accuracy. By constructing a large-scale, format-rich training corpus (open-source, real-world PDFs, and synthetic OCR data) and applying entropy-based filtering plus per-format rewards, FD-RL achieves state-of-the-art end-to-end performance on OmniDocBench with an overall score of 90.41 and strong gains on formulas and tables. The approach demonstrates that aligning optimization objectives with format integrity yields robust cross-domain parsing, enabling more reliable document understanding and extraction in real-world settings.

Abstract

Reading text from images or scanned documents via OCR models has been a longstanding focus of researchers. Intuitively, text reading is perceived as a straightforward perceptual task, and existing work primarily focuses on constructing enriched data engineering to enhance SFT capabilities. In this work, we observe that even advanced OCR models exhibit significantly higher entropy in formatted text (\emph{e.g.}, formula, table, etc.) compared to plain text, often by an order of magnitude. These statistical patterns reveal that advanced OCR models struggle with high output uncertainty when dealing with format sensitive document, suggesting that reasoning over diverse reading pathways may improve OCR performance. To address this, we propose format decoupled reinforcement learning (FD-RL), which leverages high-entropy patterns for targeted optimization. Our approach employs entropy-based data filtration strategy to identify format-intensive instances, and adopt format decoupled rewards tailored to different format types, enabling format-level validation rather than token-level memorization. FD-RL achieves an average score of 90.41 on OmniDocBench, setting a new record for end-to-end models on this highly popular benchmark. More importantly, we conduct comprehensive ablation studies over data, training, filtering, and rewarding strategies, thoroughly validating their effectiveness.

Figures (5)

• Figure 1: Document samples with a high proportion of formatted text exhibit high entropy properties. We divide the document content into plain text and formatted text. The horizontal axis represents the proportion of formatted text to the total document content, and the vertical axis represents the ratio of token entropy. We present a representative document example for each type.
• Figure 2: Overview of our training data construction pipeline. We construct the dataset via (1) open-source dataset quality enhancement, (2) real-world PDF construction, and (3) synthetic OCR data generation, yielding a large-scale dataset covering nine common document categories: notes, financial reports, slides, exam papers, synthetic data, magazines, academic papers, books, and newspapers.
• Figure 3: Overview of the FD-RL training pipeline. Our approach comprises two stages: Stage 1 (SFT) trains FD-RL(SFT) on large-scale document data via supervised fine-tuning, freezing the vision encoder and projector while updating the language model. Stage 2 (RL) introduces two innovations: (1) Entropy-based Data Filtration Strategy that filters data through type-based filtering (enriching structured data and balancing languages) and entropy-based filtering (retaining high-entropy samples), and (2) Format Decoupled RL using separately for different content types: string matching reward for plain text, expression correctness reward for formulas, and structural coherence reward for tables.
• Figure 4: Qualitative examples comparing FD-RL with baselines. FD-RL successfully follows instructions to generate HTML code and correctly extracts tables in structured format, demonstrating FD-RL's superiority in format-intensive content parsing.
• Figure 5: Qualitative comparison of Qwen3-VL-4B and FD-RL on three types of formatted content: Puzzle, Integral Formula, and Table.