Stratified Domain Adaptation: A Progressive Self-Training Approach for Scene Text Recognition

TL;DR

This paper tackles the challenge of unsupervised domain adaptation in scene text recognition by addressing large domain gaps between synthetic training data and real-world text. It introduces Stratified Domain Adaptation (StrDA), a two-stage framework that first stratifies unlabeled target data into ordered subsets using Harmonic Domain Gap Estimator (HDGE) and then performs progressive self-training on these subsets, guided by a domain-proximity score $d_i$ computed from discriminators and generators via $d_i = \frac{(1+\beta^2) d_S d_T}{\beta^2 d_S + d_T}$. The approach includes several training enhancements—label sharpening, regularization, and RandAugment—to improve pseudo-label quality, and it is evaluated on multiple STR backbones (CRNN, TRBA, ABINet) across six benchmarks and five additional datasets, showing consistent gains over vanilla self-training and competitive improvements over state-of-the-art methods. Ablation studies demonstrate that HDGE outperforms a Domain Classifier baseline, and that both the number of data partitions $n$ and the bias parameter $\beta$ significantly influence performance, with larger gains when domain proximity is leveraged from both source and target perspectives. Overall, StrDA reduces labeling costs while delivering robust cross-domain STR performance and is readily integrable with other domain adaptation techniques.

Abstract

Unsupervised domain adaptation (UDA) has become increasingly prevalent in scene text recognition (STR), especially where training and testing data reside in different domains. The efficacy of existing UDA approaches tends to degrade when there is a large gap between the source and target domains. To deal with this problem, gradually shifting or progressively learning to shift from domain to domain is the key issue. In this paper, we introduce the Stratified Domain Adaptation (StrDA) approach, which examines the gradual escalation of the domain gap for the learning process. The objective is to partition the training data into subsets so that the progressively self-trained model can adapt to gradual changes. We stratify the training data by evaluating the proximity of each data sample to both the source and target domains. We propose a novel method for employing domain discriminators to estimate the out-of-distribution and domain discriminative levels of data samples. Extensive experiments on benchmark scene-text datasets show that our approach significantly improves the performance of baseline (source-trained) STR models.

Figures (12)

• Figure 1: The overall framework of our proposed Stratified Domain Adaptation (StrDA) for scene text recognition. Our approach leverages labeled synthetic data and unlabeled real data, without human annotation. The entire process is divided into 2 stages: Domain Stratifying (partitioning the unlabeled real data into subsets satisfying \ref{['eq:domain']}) and Progressive Self-Training. $m$ represents the number of unlabeled data, and $n$ serves as the hyper-parameter in \ref{['eq:domain']}.
• Figure 2: Our architecture consists of two mapping functions, $G_T: S \rightarrow T$ and $G_S: T \rightarrow S$, along with associated adversarial discriminators, $D_S$ and $D_T$. While $G_S$ and $G_T$ are tasked with translating images from one domain to another, $D_S$ estimates the difference between an image and the data distribution of the source domain $S$, and similarly, $D_T$ does so for the target domain $T$.
• Figure 3: Ablation study on Hyper-parameter $n$ in \ref{['eq:domain']}.
• Figure 4: Ablation study on Hyper-parameter $\beta$ in \ref{['eq:hdge']}
• Figure 5: The StrDA partitions the data from the target domain into three distinct subsets, with the disparity across domains gradually rising, as shown in the image. The next two lines depict the pseudo-labels employed in the self-training process of $\mathrm{ST}$ and $\mathrm{StrDA_{HDGE}}$, respectively. The pseudo-labels generated by $\mathrm{ST}$ are prone to noise as the extent of the domain gap escalates. On the other hand, $\mathrm{StrDA_{HDGE}}$, produces pseudo-labels with higher accuracy. The STR model used for the example is TRBA.