What's Wrong with the Bottom-up Methods in Arbitrary-shape Scene Text Detection

TL;DR

This work analyzes the limitations of existing bottom-up, GCN-based arbitrary-shape text detectors, identifying error accumulation and insufficient use of relational cues as key bottlenecks. It introduces a dense overlapping text-segment framework with weakly supervised segment typing, a Graph Convolutional Network for relational reasoning, and a visual-relational fusion pipeline (LAT and FD) to produce a Graph Guided Text Region (GGTR) map, complemented by a contour-inference method (SAp) that replaces route-based geometry. The paper demonstrates substantial gains on curved and multi-oriented benchmarks (CTW1500, Total-Text, ICDAR2015, MSRA-TD500, MLT2017), arguing that bottom-up methods can outperform top-down approaches when equipped with effective false-positive/false-negative suppression and long-range contour reasoning. The approach achieves state-of-the-art performance, and ablation studies show the synergistic benefits of combining node classification with GGTR-guided fusion and shape-approximation. Overall, the work provides a practical, efficient, and robust bottom-up framework that leverages visual-relational reasoning to handle arbitrary-shaped text detection.

Abstract

The latest trend in the bottom-up perspective for arbitrary-shape scene text detection is to reason the links between text segments using Graph Convolutional Network (GCN). Notwithstanding, the performance of the best performing bottom-up method is still inferior to that of the best performing top-down method even with the help of GCN. We argue that this is not mainly caused by the limited feature capturing ability of the text proposal backbone or GCN, but by their failure to make a full use of visual-relational features for suppressing false detection, as well as the sub-optimal route-finding mechanism used for grouping text segments. In this paper, we revitalize the classic text detection frameworks by aggregating the visual-relational features of text with two effective false positive/negative suppression mechanisms. First, dense overlapping text segments depicting the `characterness' and `streamline' of text are generated for further relational reasoning and weakly supervised segment classification. Here, relational graph features are used for suppressing false positives/negatives. Then, to fuse the relational features with visual features, a Location-Aware Transfer (LAT) module is designed to transfer text's relational features into visual compatible features with a Fuse Decoding (FD) module to enhance the representation of text regions for the second step suppression. Finally, a novel multiple-text-map-aware contour-approximation strategy is developed, instead of the widely-used route-finding process. Experiments conducted on five benchmark datasets, i.e., CTW1500, Total-Text, ICDAR2015, MSRA-TD500, and MLT2017 demonstrate that our method outperforms the state-of-the-art performance when being embedded in a classic text detection framework, which revitalises the superb strength of the bottom-up methods.

Figures (9)

• Figure 1: The error accumulation problem of existing bottom-up approaches (top) and our solution (bottom). Our Graph Guided Text Region fuses relational features with visual features and jointly determines the text area, which reduces false detections. The segment type prediction module further rectifies false detections through excavating the "characterness" and connectivity of text segments. The final output shows that false detections such as missing parts and text-like objects have been suppressed.
• Figure 2: Route finding may produce suboptimal visiting orders when there are too many contour points. The resultant crossing lines are caused by incorrect visiting order of intermediate points. These crossing lines may lead to self-intersecting contours and result in detection failures.
• Figure 3: The overall structure of our network. The "1/2,64", "1/4,128",... and "1/16,512" indicate the scale ratio and the channel number of the input image. In the training flow, the TCL map, GGTR map, geometric features, link prediction and node classification are guided by the loss function. In testing flow, the GGTR map and weakly supervised node classification results are used in our proposed FPNS strategies to rectify text segments. The proposed SAp is used to obtain the final contour of text segments.
• Figure 4: The three types of text segments. All of these text segments are designed to have overlap with each other with a width between 2-6 pixels.
• Figure 5: The results of weakly supervised annotation (1st row) and the ground truth (2nd row). The Interval Segments and Char Segments are shown with different colors. The GCN now has the ability to classify the type of segments, which will benefit the following FPNS step. In these figures, Non-Text Segments are not displayed for clarity.