UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis

TL;DR

UniHDSA tackles hierarchical document structure analysis by recasting page-level and document-level subtasks as relation prediction problems within a single unified label space. The authors implement a two-stage, multimodal Transformer-based system that learns to detect page objects, determine reading order, and reconstruct cross-page hierarchies, all within a single end-to-end model. Empirical results show state-of-the-art performance on Comp-HRDoc and competitive results on DocLayNet, with extensive ablations confirming the benefits of unified modeling, cross-level relations, and semantic features. The work advances scalable, end-to-end HDSA and provides open-source configurations for replication and further research.

Abstract

Document structure analysis, aka document layout analysis, is crucial for understanding both the physical layout and logical structure of documents, serving information retrieval, document summarization, knowledge extraction, etc. Hierarchical Document Structure Analysis (HDSA) specifically aims to restore the hierarchical structure of documents created using authoring software with hierarchical schemas. Previous research has primarily followed two approaches: one focuses on tackling specific subtasks of HDSA in isolation, such as table detection or reading order prediction, while the other adopts a unified framework that uses multiple branches or modules, each designed to address a distinct task. In this work, we propose a unified relation prediction approach for HDSA, called UniHDSA, which treats various HDSA sub-tasks as relation prediction problems and consolidates relation prediction labels into a unified label space. This allows a single relation prediction module to handle multiple tasks simultaneously, whether at a page-level or document-level structure analysis. To validate the effectiveness of UniHDSA, we develop a multimodal end-to-end system based on Transformer architectures. Extensive experimental results demonstrate that our approach achieves state-of-the-art performance on a hierarchical document structure analysis benchmark, Comp-HRDoc, and competitive results on a large-scale document layout analysis dataset, DocLayNet, effectively illustrating the superiority of our method across all sub-tasks. The Comp-HRDoc benchmark and UniHDSA's configurations are publicly available at https://github.com/microsoft/CompHRDoc.

Figures (10)

• Figure 1: Comparison between previous state-of-the-art approach and our proposed method. (a) Detect-Order-Construct employs a multi-stage tree construction based approach, designing specific modules for each sub-task, which introduces cascading errors. (b) Our approach simplifies the process into two stages, unifying sub-tasks within page-level and document-level structure analysis using a unified label space. $T_i$ denotes Text-line queries, $G_i$ denotes Graphical object queries, $L_i$ denotes Logical role queries, $B_i$ denotes Text-Block queries. The purple grids illustrate intra-region relationships, the green grids represent inter-region relationships, and the orange grids signify logical role relationships.
• Figure 2: Overview of our proposed multimodal end-to-end system based on UniHDSA for hierarchical document structure analysis.
• Figure 3: An example of our page-level and document-level problem definition for hierarchical document structure analysis. Purple arrow: intra-region relationship; green arrow: inter-region relationship; orange arrow: logical role relationship. Best viewed in color. For visual clarity, random relationships are omitted.
• Figure 4: Overall architecture of our page-level structure analysis module. The different colors of the cells represent distinct types of queries. Here, $L_f$, $L_t$, and $L_r$ denote the length of multi-scale features, the number of text lines, and the number of predefined logical role queries, respectively. $D$ represents the embedding dimension, and $K$ is the number of selected graphical object proposals.
• Figure 5: The page-level unified label space in UniHDSA. $T_i$ denotes Text-line queries, $G_i$ denotes Graphical object queries, and $L_i$ denotes Logical role queries. The purple grids illustrate intra-region relationships, the green grids represent inter-region relationships, and the orange grids signify logical role relationships.