Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition

Takaya Kawakatsu

TL;DR

MuTabNet tackles end-to-end table recognition by coupling a ResNet-based encoder with a multi-cell decoder for content and a separate HTML decoder for structure, enhanced by 2D positional encoding and local attention. It introduces bidirectional mutual learning to enforce attention to both previous and following cells, while a multi-cell decoder leverages neighbor-cell information. Across FinTabNet and PubTabNet, MuTabNet achieves competitive TEDS scores without external OCR, including strong performance on long tables, demonstrating practical viability for downstream LLM-based knowledge processing. This work advances robust, end-to-end table understanding with reduced reliance on OCR and improved efficiency, enabling better integration with scientific knowledge extraction pipelines.

Abstract

Extracting table contents from documents such as scientific papers and financial reports and converting them into a format that can be processed by large language models is an important task in knowledge information processing. End-to-end approaches, which recognize not only table structure but also cell contents, achieved performance comparable to state-of-the-art models using external character recognition systems, and have potential for further improvements. In addition, these models can now recognize long tables with hundreds of cells by introducing local attention. However, the models recognize table structure in one direction from the header to the footer, and cell content recognition is performed independently for each cell, so there is no opportunity to retrieve useful information from the neighbor cells. In this paper, we propose a multi-cell content decoder and bidirectional mutual learning mechanism to improve the end-to-end approach. The effectiveness is demonstrated on two large datasets, and the experimental results show comparable performance to state-of-the-art models, even for long tables with large numbers of cells.

Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition

TL;DR

MuTabNet tackles end-to-end table recognition by coupling a ResNet-based encoder with a multi-cell decoder for content and a separate HTML decoder for structure, enhanced by 2D positional encoding and local attention. It introduces bidirectional mutual learning to enforce attention to both previous and following cells, while a multi-cell decoder leverages neighbor-cell information. Across FinTabNet and PubTabNet, MuTabNet achieves competitive TEDS scores without external OCR, including strong performance on long tables, demonstrating practical viability for downstream LLM-based knowledge processing. This work advances robust, end-to-end table understanding with reduced reliance on OCR and improved efficiency, enabling better integration with scientific knowledge extraction pipelines.

Abstract

Extracting table contents from documents such as scientific papers and financial reports and converting them into a format that can be processed by large language models is an important task in knowledge information processing. End-to-end approaches, which recognize not only table structure but also cell contents, achieved performance comparable to state-of-the-art models using external character recognition systems, and have potential for further improvements. In addition, these models can now recognize long tables with hundreds of cells by introducing local attention. However, the models recognize table structure in one direction from the header to the footer, and cell content recognition is performed independently for each cell, so there is no opportunity to retrieve useful information from the neighbor cells. In this paper, we propose a multi-cell content decoder and bidirectional mutual learning mechanism to improve the end-to-end approach. The effectiveness is demonstrated on two large datasets, and the experimental results show comparable performance to state-of-the-art models, even for long tables with large numbers of cells.
Paper Structure (27 sections, 10 equations, 1 figure, 6 tables)

This paper contains 27 sections, 10 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Encoder
  5. Decoder
  6. Local Attention
  7. Positional Encoding
  8. Mutual Learning
  9. Metrics
  10. Proposal
  11. Encoder
  12. HTML Decoder
  13. Cell Decoder
  14. Bidirectional Mutual Learning
  15. Experiments
  16. ...and 12 more sections

Figures (1)

  • Figure 1: Proposed network architecture.