SVTR: Scene Text Recognition with a Single Visual Model

TL;DR

The paper tackles the complexity of traditional two-block scene text recognizers by proposing SVTR, a single visual model that uses patch-wise tokenization and a multi-stage mixing backbone to capture both local stroke-like features and long-range inter-character dependencies. It introduces global and local mixing blocks, a progressive patch embedding scheme, and merging/combining operations to build a multi-scale, multi-grained representation, followed by a parallel linear predictor for transcription. Extensive experiments on English and Chinese benchmarks show SVTR-L achieving competitive or state-of-the-art accuracy with faster inference, while SVTR-T offers a compact, efficient alternative; ablation studies validate the design choices. The approach demonstrates strong cross-lingual performance with a simpler, faster architecture, suggesting practical viability for real-time and multilingual scene text recognition.

Abstract

Dominant scene text recognition models commonly contain two building blocks, a visual model for feature extraction and a sequence model for text transcription. This hybrid architecture, although accurate, is complex and less efficient. In this study, we propose a Single Visual model for Scene Text recognition within the patch-wise image tokenization framework, which dispenses with the sequential modeling entirely. The method, termed SVTR, firstly decomposes an image text into small patches named character components. Afterward, hierarchical stages are recurrently carried out by component-level mixing, merging and/or combining. Global and local mixing blocks are devised to perceive the inter-character and intra-character patterns, leading to a multi-grained character component perception. Thus, characters are recognized by a simple linear prediction. Experimental results on both English and Chinese scene text recognition tasks demonstrate the effectiveness of SVTR. SVTR-L (Large) achieves highly competitive accuracy in English and outperforms existing methods by a large margin in Chinese, while running faster. In addition, SVTR-T (Tiny) is an effective and much smaller model, which shows appealing speed at inference. The code is publicly available at https://github.com/PaddlePaddle/PaddleOCR.

Figures (6)

• Figure 1: (a) CNN-RNN based models. (b) Encoder-Decoder models. MHSA and MHA denote multi-head self-attention and multi-head attention, respectively. (c) Vision-Language models. (d) Our SVTR, which recognizes scene text with a single visual model and enjoys efficient, accurate and cross-lingual versatile.
• Figure 2: Overall architecture of the proposed SVTR. It is a three-stage height progressively decreased network. In each stage, a series of mixing blocks are carried out and followed by a merging or combining operation. At last, the recognition is conducted by a linear prediction.
• Figure 3: (a) The linear projection in ViT dosovitskiy2020vit. (b) Our progressive overlapping patch embedding.
• Figure 4: Illustration of (a) global mixing and (b) local mixing.
• Figure 5: Accuracy-parameter (M) and Accuracy-speed (ms) plots of different models on IC15.