CPN: Complementary Proposal Network for Unconstrained Text Detection

TL;DR

CPN tackles unconstrained scene text detection by uniting segmentation-like semantic cues and anchor-based geometric proposals in two parallel networks. The Deformable Morphology Semantic Network provides efficient semantic proposals through differentiable morphological operations, while the Balanced Region Proposal Network supplies geometric proposals with oriented anchors; an Interleaved Feature Attention module fuses these signals for mutual supervision. Across five benchmarks, CPN achieves new or competitive state-of-the-art results with favorable speed, illustrating that complementary proposal generation and cross-branch interaction substantially improve recall and precision for curved, long, and multilingual text. The work demonstrates the practical value of end-to-end trainable, dual-proposal architectures for complex text detection tasks.

Abstract

Existing methods for scene text detection can be divided into two paradigms: segmentation-based and anchor-based. While Segmentation-based methods are well-suited for irregular shapes, they struggle with compact or overlapping layouts. Conversely, anchor-based approaches excel for complex layouts but suffer from irregular shapes. To strengthen their merits and overcome their respective demerits, we propose a Complementary Proposal Network (CPN) that seamlessly and parallelly integrates semantic and geometric information for superior performance. The CPN comprises two efficient networks for proposal generation: the Deformable Morphology Semantic Network, which generates semantic proposals employing an innovative deformable morphological operator, and the Balanced Region Proposal Network, which produces geometric proposals with pre-defined anchors. To further enhance the complementarity, we introduce an Interleaved Feature Attention module that enables semantic and geometric features to interact deeply before proposal generation. By leveraging both complementary proposals and features, CPN outperforms state-of-the-art approaches with significant margins under comparable computation cost. Specifically, our approach achieves improvements of 3.6%, 1.3% and 1.0% on challenging benchmarks ICDAR19-ArT, IC15, and MSRA-TD500, respectively. Code for our method will be released.

Figures (6)

• Figure 1: Comparison between existing detectors and our proposed CPN. The input image is shown in (\ref{['fig:intro_img']}). Segmentation based methods tend to produce false positives and negatives while facing compact layouts as illustrated in (\ref{['fig:intro_sem']}). Anchor based approaches struggle for texts of large aspect ratio as shown in (\ref{['fig:intro_geo']}). Our CPN addresses the above issues by complementary proposals and features, with results given in (\ref{['fig:intro_ours']}). True positive results are colored in green, while the false boxes are indicated in red.
• Figure 2: The pipeline of the proposed Complementary Proposal Network (CPN). Given an input image, multi-scale FPN features are extracted and fed to two parallel networks (Deformable MSN and Balanced RPN) for generating complementary semantic and geometric proposals. An Interleaved Feature Attention (IFA) module is designed to promote interaction between branch features, encouraging them to capture more spatial and scale-aware information. Features for the merged proposals are then identified with RRoI align before passing to the RoI head, where final text boxes and masks are generated.
• Figure 3: Illustration of the proposed Deformable Morphology Semantic Network (Deformable MSN) for semantic proposals. It predicts a text erosion map $S$ and a structuring kernel map $D$, followed by a deformable dilation layer to produce text candidate regions, and thereafter the corresponding oriented proposals.
• Figure 4: Detailed structure of IFA module with spatial and scale interleaved attentions for the Deformable MSN branch, and the Balanced RPN branch shares the same design.
• Figure 5: Recall vs. IoU overlap ratio on the CTW1500 test set and IC17-MLT validation set. Rotated bounding boxes are applied while computing IoU on CTW1500.