GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarking

TL;DR

This work tackles the recognition bottleneck in video text spotting by proposing GoMatching++, a parameter- and data-efficient method that freezes an image text spotter and adds a lightweight rescoring head plus a Long-Short-Term Matching (LST-Matcher) tracker. The rescoring mechanism bridges the image–video domain gap, while LST-Matcher fuses short- and long-term cues for robust tracking; several architectural variants are explored to minimize trainable parameters. The authors introduce ArTVideo, a curved-text-in-video benchmark with 60 videos and detailed polygon/mask annotations to address curved text gaps in existing datasets. Empirical results on ICDAR15-video, BOVText, and DSText show GoMatching++ achieving state-of-the-art performance with substantially reduced training cost, and ArTVideo enables comprehensive analysis of curved-text handling in VTS. Overall, the approach offers a practical, scalable pathway to high-performance VTS and provides a benchmark to catalyze future research on arbitrary-shaped text in video.

Abstract

Video text spotting (VTS) extends image text spotting (ITS) by adding text tracking, significantly increasing task complexity. Despite progress in VTS, existing methods still fall short of the performance seen in ITS. This paper identifies a key limitation in current video text spotters: limited recognition capability, even after extensive end-to-end training. To address this, we propose GoMatching++, a parameter- and data-efficient method that transforms an off-the-shelf image text spotter into a video specialist. The core idea lies in freezing the image text spotter and introducing a lightweight, trainable tracker, which can be optimized efficiently with minimal training data. Our approach includes two key components: (1) a rescoring mechanism to bridge the domain gap between image and video data, and (2) the LST-Matcher, which enhances the frozen image text spotter's ability to handle video text. We explore various architectures for LST-Matcher to ensure efficiency in both parameters and training data. As a result, GoMatching++ sets new performance records on challenging benchmarks such as ICDAR15-video, DSText, and BOVText, while significantly reducing training costs. To address the lack of curved text datasets in VTS, we introduce ArTVideo, a new benchmark featuring over 30% curved text with detailed annotations. We also provide a comprehensive statistical analysis and experimental results for ArTVideo. We believe that GoMatching++ and the ArTVideo benchmark will drive future advancements in video text spotting. The source code, models and dataset are publicly available at https://github.com/Hxyz-123/GoMatching.

Figures (9)

• Figure 1: (a) 'Gap between Spot. & Det.': the gap between spotting and detection F1-score. As the spotting task involves recognizing the results of the detection process, the detection score is indeed the upper bound of spotting performance. The larger the gap, the poorer the recognition ability. Compared to the ITS model (Deepsolo ye2023deepsolo), the VTS model (TransDETR wu2024end) presents unsatisfactory image-level text spotting F1-scores, which lag far behind its detection performance, especially on ArTVideo with curved text. It indicates recognition capability is a main bottleneck in the VTS model. (b) GoMatching++ outperforms TransDETR by over 9.68 MOTA on ICDAR15-video while saving 301 training GPU hours and 19.4GB memory.
• Figure 2: A. The overall architecture of GoMatching++. The frozen image text spotter provides text spotting results for frames. The rescoring mechanism utilizes both instance scores from the image text spotter and a trainable rescoring head to reduce performance degradation due to the domain gap. LST-Matcher assigns IDs to text instances based on the queries in long-short term frames. The yellow star sign '$\bigstar$' indicates the final output. B. Four architectural designs of LST-Matcher. As ST-Matcher and LT-Matcher in LST-Matcher have the same architectures, only one is shown for simplicity. '*' indicates that ST-Matcher and LT-Matcher shared the same network parameters.
• Figure 3: The inference pipeline of LST-Matcher, which is a two-stage association process: (1) ST-Matcher associates the instances with trajectories in previous frames as denoted by blue lines. (2) LT-Matcher associates the remaining unmatched instances by utilizing other trajectories in history frames as denoted by red lines.
• Figure 4: Visual examples from ArTVideo training set. The straight and curved text are labeled with quadrilaterals and polygons, respectively. The same background color in different frames denotes the same instance.
• Figure 5: Statistics of ArTVideo. (a) and (b) show the distribution of various types of text instance numbers in each frame and the distribution of the text length of each instance, respectively. (c) presents the word cloud of text annotations in ArTVideo.