Hierarchical Modeling Approach to Fast and Accurate Table Recognition

TL;DR

This work tackles fast and accurate table image recognition by introducing a hierarchical multi-task model that uses non-causal attention to capture global table structure and a parallel inference mechanism for cell contents. It adds an HTML refiner to enable cross-cell information flow and a cell decoder that decodes all cell contents in parallel, significantly speeding up inference. Across FinTabNet and PubTabNet, the approach achieves state-of-the-art or competitive performance on structural and total TEDS while reducing inference time substantially. The study also provides ablation and case studies to explain how dense, multi-task features contribute to improved reading of long and complex tables, highlighting practical impact for scalable document understanding.

Abstract

The extraction and use of diverse knowledge from numerous documents is a pressing challenge in intelligent information retrieval. Documents contain elements that require different recognition methods. Table recognition typically consists of three subtasks, namely table structure, cell position and cell content recognition. Recent models have achieved excellent recognition with a combination of multi-task learning, local attention, and mutual learning. However, their effectiveness has not been fully explained, and they require a long period of time for inference. This paper presents a novel multi-task model that utilizes non-causal attention to capture the entire table structure, and a parallel inference algorithm for faster cell content inference. The superiority is demonstrated both visually and statistically on two large public datasets.

Figures (2)

• Figure 1: Proposed network architecture with a refiner and parallel inference algorithm to improve cell content recognition.
• Figure 2: Attention map from the cell decoder with the maximum shown in white and the cell bounding boxes from the refiner.