Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One Model for Two Tasks: Cooperatively Recognizing and Recovering Low-Resolution Scene Text Images by Iterative Mutual Guidance

Minyi Zhao, Yang Wang, Jihong Guan, Shuigeng Zhou

TL;DR

A novel method called IMAGE is proposed to effectively recognize and recover LR scene text images simultaneously and develops an iterative mutual guidance mechanism, with which the STR model provides high-level semantic information as clue to the STISR model for better super-resolution.

Abstract

Scene text recognition (STR) from high-resolution (HR) images has been significantly successful, however text reading on low-resolution (LR) images is still challenging due to insufficient visual information. Therefore, recently many scene text image super-resolution (STISR) models have been proposed to generate super-resolution (SR) images for the LR ones, then STR is done on the SR images, which thus boosts recognition performance. Nevertheless, these methods have two major weaknesses. On the one hand, STISR approaches may generate imperfect or even erroneous SR images, which mislead the subsequent recognition of STR models. On the other hand, as the STISR and STR models are jointly optimized, to pursue high recognition accuracy, the fidelity of SR images may be spoiled. As a result, neither the recognition performance nor the fidelity of STISR models are desirable. Then, can we achieve both high recognition performance and good fidelity? To this end, in this paper we propose a novel method called IMAGE (the abbreviation of Iterative MutuAl GuidancE) to effectively recognize and recover LR scene text images simultaneously. Concretely, IMAGE consists of a specialized STR model for recognition and a tailored STISR model to recover LR images, which are optimized separately. And we develop an iterative mutual guidance mechanism, with which the STR model provides high-level semantic information as clue to the STISR model for better super-resolution, meanwhile the STISR model offers essential low-level pixel clue to the STR model for more accurate recognition. Extensive experiments on two LR datasets demonstrate the superiority of our method over the existing works on both recognition performance and super-resolution fidelity.

One Model for Two Tasks: Cooperatively Recognizing and Recovering Low-Resolution Scene Text Images by Iterative Mutual Guidance

TL;DR

A novel method called IMAGE is proposed to effectively recognize and recover LR scene text images simultaneously and develops an iterative mutual guidance mechanism, with which the STR model provides high-level semantic information as clue to the STISR model for better super-resolution.

Abstract

Scene text recognition (STR) from high-resolution (HR) images has been significantly successful, however text reading on low-resolution (LR) images is still challenging due to insufficient visual information. Therefore, recently many scene text image super-resolution (STISR) models have been proposed to generate super-resolution (SR) images for the LR ones, then STR is done on the SR images, which thus boosts recognition performance. Nevertheless, these methods have two major weaknesses. On the one hand, STISR approaches may generate imperfect or even erroneous SR images, which mislead the subsequent recognition of STR models. On the other hand, as the STISR and STR models are jointly optimized, to pursue high recognition accuracy, the fidelity of SR images may be spoiled. As a result, neither the recognition performance nor the fidelity of STISR models are desirable. Then, can we achieve both high recognition performance and good fidelity? To this end, in this paper we propose a novel method called IMAGE (the abbreviation of Iterative MutuAl GuidancE) to effectively recognize and recover LR scene text images simultaneously. Concretely, IMAGE consists of a specialized STR model for recognition and a tailored STISR model to recover LR images, which are optimized separately. And we develop an iterative mutual guidance mechanism, with which the STR model provides high-level semantic information as clue to the STISR model for better super-resolution, meanwhile the STISR model offers essential low-level pixel clue to the STR model for more accurate recognition. Extensive experiments on two LR datasets demonstrate the superiority of our method over the existing works on both recognition performance and super-resolution fidelity.
Paper Structure (22 sections, 11 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 22 sections, 11 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Scene text image super-resolution
  4. Scene text recognition
  5. Differences between related works and our method
  6. Methodology
  7. Overview
  8. Super-resolution model
  9. Text recognition model
  10. Iterative mutual guidance
  11. Loss functions
  12. Overall training procedure
  13. Performance Evaluation
  14. Datasets and metrics
  15. Implementation details
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Schematic illustration of existing works: (a) STR methods and (b) STISR methods, and (c) our method IMAGE that employs two models to do recognition and recovery respectively, which are optimized separately, but mutually provide guidance clues to each other in an iterative way. The rightmost character strings are the recognition results, where red characters are incorrectly recognized, and black ones are correctly recognized.
  • Figure 2: The architecture of our method IMAGE.
  • Figure 3: Examples of generated images and recognition results. Here, GT is ground truth text. Red/black characters are incorrectly/correctly recognized. Texts below pictures in the LR, TG, TPGSR, C3-STISR and HR columns are recognized by SVTR.
  • Figure 4: Examples of the intermediate recognition results and SR images of IMAGE-2. Here, GT indicates ground truth. Red/black characters indicate incorrectly/correctly recognized.