Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Best Practices for a Handwritten Text Recognition System

George Retsinas, Giorgos Sfikas, Basilis Gatos, Christophoros Nikou

TL;DR

Handwritten Text Recognition (HTR) remains challenging due to writer variability and ambiguity in sequence alignment. The authors propose a set of best-practice guidelines for a CNN+BiLSTM HTR system trained with the Connectionist Temporal Classification (CTC) loss, including (i) aspect-ratio-preserving preprocessing with padding, (ii) column-wise max-pooling flattening between CNN and the recurrent head, and (iii) an auxiliary CTC shortcut branch trained in parallel. They validate these choices on IAM and RIMES with both line- and word-level, lexicon-free decoding, reporting CER and WER improvements and ablations demonstrating each component's contribution. The results are competitive with state-of-the-art methods while maintaining a relatively simple architecture, and the authors provide open-source code to facilitate adoption.

Abstract

Handwritten text recognition has been developed rapidly in the recent years, following the rise of deep learning and its applications. Though deep learning methods provide notable boost in performance concerning text recognition, non-trivial deviation in performance can be detected even when small pre-processing or architectural/optimization elements are changed. This work follows a ``best practice'' rationale; highlight simple yet effective empirical practices that can further help training and provide well-performing handwritten text recognition systems. Specifically, we considered three basic aspects of a deep HTR system and we proposed simple yet effective solutions: 1) retain the aspect ratio of the images in the preprocessing step, 2) use max-pooling for converting the 3D feature map of CNN output into a sequence of features and 3) assist the training procedure via an additional CTC loss which acts as a shortcut on the max-pooled sequential features. Using these proposed simple modifications, one can attain close to state-of-the-art results, while considering a basic convolutional-recurrent (CNN+LSTM) architecture, for both IAM and RIMES datasets. Code is available at https://github.com/georgeretsi/HTR-best-practices/.

Best Practices for a Handwritten Text Recognition System

TL;DR

Handwritten Text Recognition (HTR) remains challenging due to writer variability and ambiguity in sequence alignment. The authors propose a set of best-practice guidelines for a CNN+BiLSTM HTR system trained with the Connectionist Temporal Classification (CTC) loss, including (i) aspect-ratio-preserving preprocessing with padding, (ii) column-wise max-pooling flattening between CNN and the recurrent head, and (iii) an auxiliary CTC shortcut branch trained in parallel. They validate these choices on IAM and RIMES with both line- and word-level, lexicon-free decoding, reporting CER and WER improvements and ablations demonstrating each component's contribution. The results are competitive with state-of-the-art methods while maintaining a relatively simple architecture, and the authors provide open-source code to facilitate adoption.

Abstract

Handwritten text recognition has been developed rapidly in the recent years, following the rise of deep learning and its applications. Though deep learning methods provide notable boost in performance concerning text recognition, non-trivial deviation in performance can be detected even when small pre-processing or architectural/optimization elements are changed. This work follows a ``best practice'' rationale; highlight simple yet effective empirical practices that can further help training and provide well-performing handwritten text recognition systems. Specifically, we considered three basic aspects of a deep HTR system and we proposed simple yet effective solutions: 1) retain the aspect ratio of the images in the preprocessing step, 2) use max-pooling for converting the 3D feature map of CNN output into a sequence of features and 3) assist the training procedure via an additional CTC loss which acts as a shortcut on the max-pooled sequential features. Using these proposed simple modifications, one can attain close to state-of-the-art results, while considering a basic convolutional-recurrent (CNN+LSTM) architecture, for both IAM and RIMES datasets. Code is available at https://github.com/georgeretsi/HTR-best-practices/.
Paper Structure (14 sections, 1 equation, 3 figures, 3 tables)

This paper contains 14 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed HTR System
  4. Preprocessing
  5. Network Architecture
  6. Convolutional Backbone:
  7. Flattening Operation:
  8. Recurrent Head:
  9. Training Scheme
  10. CTC shortcut:
  11. Experimental Evaluation
  12. Ablation Study
  13. Comparison to State-of-the-Art Systems
  14. Conclusions

Figures (3)

  • Figure 1: Overview of the DNN architecture. Apart from the CNN backbone, the recurrent head, we also depict the auxiliary CTC shortcut branch which will be the core component of the proposed training modification. Furthermore, we visualize the column-wise max-pooling operation that is performed between the CNN backbone and the recurrent head.
  • Figure 2: Behavior of HTR performance in terms of loss value with and without the extra CTC shortcut branch during the training phase. Reported curves correspond to the proposed line-level HTR system trained on the IAM dataset.
  • Figure 3: Behavior of HTR performance in terms of CER (left) and WER (right) metrics with and without the extra CTC shortcut branch during the training phase. Reported curves correspond to the proposed line-level HTR system, trained on IAM dataset and evaluated on the validation set.