Out of Length Text Recognition with Sub-String Matching

TL;DR

This work tackles Out of Length (OOL) text recognition by introducing SMTR, a sub-String Matching-based method that iteratively recognizes text of arbitrary length using two cross-attention modules. Trained solely on short-text data, SMTR achieves strong performance on standard short-text benchmarks and demonstrates a clear advantage on the proposed Long Text Benchmark (LTB), aided by Regularization Training to distinguish similar substrings and Inference Augmentation to mitigate repeated-substring confusion and improve efficiency. The authors also establish the Long Text Benchmark to evaluate long-text capabilities and show SMTR's effectiveness across English and Chinese datasets, highlighting significant gains over state-of-the-art baselines in long-text scenarios. While iterative decoding yields accuracy benefits, the approach accepts a trade-off with decoding speed, which the authors plan to address in future work.

Abstract

Scene Text Recognition (STR) methods have demonstrated robust performance in word-level text recognition. However, in real applications the text image is sometimes long due to detected with multiple horizontal words. It triggers the requirement to build long text recognition models from readily available short (i.e., word-level) text datasets, which has been less studied previously. In this paper, we term this task Out of Length (OOL) text recognition. We establish the first Long Text Benchmark (LTB) to facilitate the assessment of different methods in long text recognition. Meanwhile, we propose a novel method called OOL Text Recognition with sub-String Matching (SMTR). SMTR comprises two cross-attention-based modules: one encodes a sub-string containing multiple characters into next and previous queries, and the other employs the queries to attend to the image features, matching the sub-string and simultaneously recognizing its next and previous character. SMTR can recognize text of arbitrary length by iterating the process above. To avoid being trapped in recognizing highly similar sub-strings, we introduce a regularization training to compel SMTR to effectively discover subtle differences between similar sub-strings for precise matching. In addition, we propose an inference augmentation strategy to alleviate confusion caused by identical sub-strings in the same text and improve the overall recognition efficiency. Extensive experimental results reveal that SMTR, even when trained exclusively on short text, outperforms existing methods in public short text benchmarks and exhibits a clear advantage on LTB. Code: https://github.com/Topdu/OpenOCR.

Figures (12)

• Figure 1: Attention-based methods like NRTR Sheng2019nrtr and PARSeq BautistaA22PARSeq perform well in short text recognition, while the CTC-based FocalSVTR (CTC) and the length-insensitive LISTER iccv2023lister outperform them in long text but worse in short text. Our proposed SMTR gets superior performance in both short and long text recognition.
• Figure 2: Overview of SMTR. SMTR is a lightweight recognizer consisting of two cross-attention-based modules, i.e., sub-string encoder and sub-string matcher. $S$ and $\tilde{S}$ denote a sub-string and it regularized counterpart. $Y_s$ denotes the next-previous character set of the sub-string. [R] denotes a randomly selected character from the character set.
• Figure 3: Attention maps of SMTR w/o (left) and w/ (right) the proposed regularization training, which rectifies recognition errors (red) caused by similar sub-strings (yellow).
• Figure 4: Illustration of SMTR base inference process, where [E] means the end token for inference termination.
• Figure 5: Illustration of inference augmentation (IA) for long text recognition. Text spaces are manually inserted.