Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPRINT: Script-agnostic Structure Recognition in Tables

Dhruv Kudale, Badri Vishal Kasuba, Venkatapathy Subramanian, Parag Chaudhuri, Ganesh Ramakrishnan

TL;DR

This work tackles multilingual table structure recognition (TSR) by proposing SPRINT, a script-agnostic Im2Seq model that predicts compact OTSL sequences using a GCA-based encoder and a Transformer decoder while downsampling inputs to $128 \times 128$ to blur script-specific cues. A Table Grid Estimator (TATR $v1.1$) provides the table’s row and column counts ($R$ and $C$), enabling alignment of the predicted OTSL into a valid $R \times (C+1)$ matrix and subsequent HTML conversion; this combination yields fast, cross-script TSR without heavy language-specific training. Evaluations on PubTabNet, FinTabNet, and PubTables-1M show competitive TED-S scores relative to HTML-based baselines while achieving lower latency, and the MUSTARD dataset demonstrates substantial improvements in non-English scripts (average ~$11.12\%$ TEDS-S) over prior methods. The authors release MUSTARD and outline future work to integrate grid-detected cells with predicted structures for end-to-end table reconstruction across diverse scripts and modalities.

Abstract

Table Structure Recognition (TSR) is vital for various downstream tasks like information retrieval, table reconstruction, and document understanding. While most state-of-the-art (SOTA) research predominantly focuses on TSR in English documents, the need for similar capabilities in other languages is evident, considering the global diversity of data. Moreover, creating substantial labeled data in non-English languages and training these SOTA models from scratch is costly and time-consuming. We propose TSR as a language-agnostic cell arrangement prediction and introduce SPRINT, Script-agnostic Structure Recognition in Tables. SPRINT uses recently introduced Optimized Table Structure Language (OTSL) sequences to predict table structures. We show that when coupled with a pre-trained table grid estimator, SPRINT can improve the overall tree edit distance-based similarity structure scores of tables even for non-English documents. We experimentally evaluate our performance across benchmark TSR datasets including PubTabNet, FinTabNet, and PubTables-1M. Our findings reveal that SPRINT not only matches SOTA models in performance on standard datasets but also demonstrates lower latency. Additionally, SPRINT excels in accurately identifying table structures in non-English documents, surpassing current leading models by showing an absolute average increase of 11.12%. We also present an algorithm for converting valid OTSL predictions into a widely used HTML-based table representation. To encourage further research, we release our code and Multilingual Scanned and Scene Table Structure Recognition Dataset, MUSTARD labeled with OTSL sequences for 1428 tables in thirteen languages encompassing several scripts at https://github.com/IITB-LEAP-OCR/SPRINT

SPRINT: Script-agnostic Structure Recognition in Tables

TL;DR

This work tackles multilingual table structure recognition (TSR) by proposing SPRINT, a script-agnostic Im2Seq model that predicts compact OTSL sequences using a GCA-based encoder and a Transformer decoder while downsampling inputs to to blur script-specific cues. A Table Grid Estimator (TATR ) provides the table’s row and column counts ( and ), enabling alignment of the predicted OTSL into a valid matrix and subsequent HTML conversion; this combination yields fast, cross-script TSR without heavy language-specific training. Evaluations on PubTabNet, FinTabNet, and PubTables-1M show competitive TED-S scores relative to HTML-based baselines while achieving lower latency, and the MUSTARD dataset demonstrates substantial improvements in non-English scripts (average ~ TEDS-S) over prior methods. The authors release MUSTARD and outline future work to integrate grid-detected cells with predicted structures for end-to-end table reconstruction across diverse scripts and modalities.

Abstract

Table Structure Recognition (TSR) is vital for various downstream tasks like information retrieval, table reconstruction, and document understanding. While most state-of-the-art (SOTA) research predominantly focuses on TSR in English documents, the need for similar capabilities in other languages is evident, considering the global diversity of data. Moreover, creating substantial labeled data in non-English languages and training these SOTA models from scratch is costly and time-consuming. We propose TSR as a language-agnostic cell arrangement prediction and introduce SPRINT, Script-agnostic Structure Recognition in Tables. SPRINT uses recently introduced Optimized Table Structure Language (OTSL) sequences to predict table structures. We show that when coupled with a pre-trained table grid estimator, SPRINT can improve the overall tree edit distance-based similarity structure scores of tables even for non-English documents. We experimentally evaluate our performance across benchmark TSR datasets including PubTabNet, FinTabNet, and PubTables-1M. Our findings reveal that SPRINT not only matches SOTA models in performance on standard datasets but also demonstrates lower latency. Additionally, SPRINT excels in accurately identifying table structures in non-English documents, surpassing current leading models by showing an absolute average increase of 11.12%. We also present an algorithm for converting valid OTSL predictions into a widely used HTML-based table representation. To encourage further research, we release our code and Multilingual Scanned and Scene Table Structure Recognition Dataset, MUSTARD labeled with OTSL sequences for 1428 tables in thirteen languages encompassing several scripts at https://github.com/IITB-LEAP-OCR/SPRINT

Paper Structure

This paper contains 21 sections, 2 equations, 10 figures, 15 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Our Methodology
  4. Experiments
  5. Datasets
  6. Image Preprocessing
  7. Training SPRINT
  8. Table Grid Estimator
  9. Results and Discussions
  10. Performance on TSR in English Documents
  11. Performance on MUSTARD
  12. Conclusion
  13. Understanding Table Structures
  14. Physical Structure
  15. Logical Structure
  16. ...and 6 more sections

Figures (10)

  • Figure 1: Recent performance trends of various Im2Seq models for TSR
  • Figure 1: Understanding Table Structure Recognition (TSR) with notions of physical and logical structures as illustrated in TableFormer tableformer
  • Figure 2: Our methodology for TSR using SPRINT
  • Figure 2: Predictions for a table image from FinTabNet dataset using TATR v1.1
  • Figure 3: Sample images from our internal dataset, MUSTARD
  • ...and 5 more figures