Arbitrary Reading Order Scene Text Spotter with Local Semantics Guidance

TL;DR

This work addresses the challenge of arbitrary reading order in scene text spotting by reframing spotting as an autoregressive recognition task guided by local semantics. It introduces SPLM to determine robust reading order start points and MAAM to perform adaptive, local grid-attentive decoding of each character, enabling end-to-end, detection-free text spotting with improved efficiency via grid sampling. The approach achieves state-of-the-art results on the challenging InverseText benchmark and strong performance on Total-Text and SCUT-CTW1500, demonstrating robustness to inverse and curved texts. The method reduces reliance on precise detections while leveraging local semantic cues, offering practical benefits for real-world scene understanding and OCR tasks.

Abstract

Scene text spotting has attracted the enthusiasm of relative researchers in recent years. Most existing scene text spotters follow the detection-then-recognition paradigm, where the vanilla detection module hardly determines the reading order and leads to failure recognition. After rethinking the auto-regressive scene text recognition method, we find that a well-trained recognizer can implicitly perceive the local semantics of all characters in a complete word or a sentence without a character-level detection module. Local semantic knowledge not only includes text content but also spatial information in the right reading order. Motivated by the above analysis, we propose the Local Semantics Guided scene text Spotter (LSGSpotter), which auto-regressively decodes the position and content of characters guided by the local semantics. Specifically, two effective modules are proposed in LSGSpotter. On the one hand, we design a Start Point Localization Module (SPLM) for locating text start points to determine the right reading order. On the other hand, a Multi-scale Adaptive Attention Module (MAAM) is proposed to adaptively aggregate text features in a local area. In conclusion, LSGSpotter achieves the arbitrary reading order spotting task without the limitation of sophisticated detection, while alleviating the cost of computational resources with the grid sampling strategy. Extensive experiment results show LSGSpotter achieves state-of-the-art performance on the InverseText benchmark. Moreover, our spotter demonstrates superior performance on English benchmarks for arbitrary-shaped text, achieving improvements of 0.7\% and 2.5\% on Total-Text and SCUT-CTW1500, respectively. These results validate our text spotter is effective for scene texts in arbitrary reading order and shape.

Figures (7)

• Figure 1: The comparison of arbitrary reading order text instances and analysis from Total-Text ch2017total of ABCNet v2 liu2021abcnet, SPTS peng2022spts and ESTextSpotter huang2023estextspotter. Our spotter can use the locally fine-grained semantics to perceive reading order without accurate detection dependency.
• Figure 2: The architecture of LSGSpotter. Image encoder refers to the aggregation of the backbone and neck. SPLM and MAAM are abbreviations of Start Point Localization Module and Multi-level Adaptive Attention Module respectively. The start point produced by SPLM is the first reference point in the MAAM.
• Figure 3: The visualization of Label Generation on different language datasets. Points in different colors represent the different text instances in (a) and (b). The red arrows in (c) show the disturbance shift of center points.
• Figure 4: Impact of reference point disturbance strategy on model performance. Without disturbance during training, some characters will be omitted in the inference stage.
• Figure 5: Qualitative results on the testing set of InverseText, Total-Text, SCUT-CTW1500 from left to right in the first line. The second line is the visualization of local grids predicted in MAAM for some challenging text instances. The color from light to deep indicates the decoding order.