Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Relation Transformer for Contextual Text Block Detection

Jiawei Wang, Shunchi Zhang, Kai Hu, Chixiang Ma, Zhuoyao Zhong, Lei Sun, Qiang Huo

TL;DR

This work tackles contextual text block detection (CTBD) in natural scenes by modeling text units as graph nodes and reading-order relations as edges. It introduces a two-component framework: an integral text detector based on DQ-DETR to extract text units and a Dynamic Relation Transformer (DRFormer) to predict and iteratively refine edges via a dual interactive decoder with deformable attention. Empirical results on SCUT-CTW-Context and ReCTS-Context show state-of-the-art performance, with ablations validating the benefits of dynamic graph structure refinement, cross-attention ordering, and relation-aware self-attention. By adopting a graph-generation perspective, the approach advances CTBD and offers potential gains for downstream scene-text understanding and NLP tasks that rely on coherent text blocks.

Abstract

Contextual Text Block Detection (CTBD) is the task of identifying coherent text blocks within the complexity of natural scenes. Previous methodologies have treated CTBD as either a visual relation extraction challenge within computer vision or as a sequence modeling problem from the perspective of natural language processing. We introduce a new framework that frames CTBD as a graph generation problem. This methodology consists of two essential procedures: identifying individual text units as graph nodes and discerning the sequential reading order relationships among these units as graph edges. Leveraging the cutting-edge capabilities of DQ-DETR for node detection, our framework innovates further by integrating a novel mechanism, a Dynamic Relation Transformer (DRFormer), dedicated to edge generation. DRFormer incorporates a dual interactive transformer decoder that deftly manages a dynamic graph structure refinement process. Through this iterative process, the model systematically enhances the graph's fidelity, ultimately resulting in improved precision in detecting contextual text blocks. Comprehensive experimental evaluations conducted on both SCUT-CTW-Context and ReCTS-Context datasets substantiate that our method achieves state-of-the-art results, underscoring the effectiveness and potential of our graph generation framework in advancing the field of CTBD.

Dynamic Relation Transformer for Contextual Text Block Detection

TL;DR

This work tackles contextual text block detection (CTBD) in natural scenes by modeling text units as graph nodes and reading-order relations as edges. It introduces a two-component framework: an integral text detector based on DQ-DETR to extract text units and a Dynamic Relation Transformer (DRFormer) to predict and iteratively refine edges via a dual interactive decoder with deformable attention. Empirical results on SCUT-CTW-Context and ReCTS-Context show state-of-the-art performance, with ablations validating the benefits of dynamic graph structure refinement, cross-attention ordering, and relation-aware self-attention. By adopting a graph-generation perspective, the approach advances CTBD and offers potential gains for downstream scene-text understanding and NLP tasks that rely on coherent text blocks.

Abstract

Contextual Text Block Detection (CTBD) is the task of identifying coherent text blocks within the complexity of natural scenes. Previous methodologies have treated CTBD as either a visual relation extraction challenge within computer vision or as a sequence modeling problem from the perspective of natural language processing. We introduce a new framework that frames CTBD as a graph generation problem. This methodology consists of two essential procedures: identifying individual text units as graph nodes and discerning the sequential reading order relationships among these units as graph edges. Leveraging the cutting-edge capabilities of DQ-DETR for node detection, our framework innovates further by integrating a novel mechanism, a Dynamic Relation Transformer (DRFormer), dedicated to edge generation. DRFormer incorporates a dual interactive transformer decoder that deftly manages a dynamic graph structure refinement process. Through this iterative process, the model systematically enhances the graph's fidelity, ultimately resulting in improved precision in detecting contextual text blocks. Comprehensive experimental evaluations conducted on both SCUT-CTW-Context and ReCTS-Context datasets substantiate that our method achieves state-of-the-art results, underscoring the effectiveness and potential of our graph generation framework in advancing the field of CTBD.
Paper Structure (32 sections, 4 equations, 3 figures, 4 tables)

This paper contains 32 sections, 4 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Scene Text Detection
  4. Text Region Detection
  5. Top-Down Methods.
  6. Bottom-Up Methods.
  7. Methodology
  8. Integral Text Detector
  9. Dynamic Relation Transformer
  10. Query Initialization.
  11. Dual Interactive Decoder.
  12. Dynamic Edge Query Refinement.
  13. Final Edge Prediction.
  14. Optimization
  15. Loss for Integral Text Detector.
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Overview of our proposed framework for contextual text block detection.
  • Figure 2: The architecture of our proposed DRFormer, consisting of a dual interactive decoder. For clarity, we only show two attention layers of Transformer decoder and omit the FFN blocks.
  • Figure 3: Qualitative comparison between our proposed baseline (top) and DRFormer (bottom). Black bounding boxes indicate word-level or character-level integral texts, i.e., graph's nodes, and brown arrows represent the reading order relationships between these integral texts, i.e., graph's edges, which finally lead to contextual text blocks outlined in green bounding boxes. Best viewed in color.