Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System

Lalita Kumari, Sukhdeep Singh, Vaibhav Varish Singh Rathore, Anuj Sharma

TL;DR

GatedLexiconNet addresses offline handwritten paragraph recognition with an end-to-end, segmentation-free approach. It integrates a gated convolutional encoder, vertical attention for internal line segmentation, parcel-end detection, and a lexicon-driven Word Beam Search decoder to produce line-by-line transcriptions from paragraph images. The model demonstrates notable improvements in character and word error rates on IAM, RIMES, and READ-2016 compared to segmentation-based baselines, validating the effectiveness of gating and lexicon-aware decoding for complex handwriting layouts. Although effective, the approach currently assumes single-column layouts and incurs substantial computation, guiding future work toward scalability and multi-column handling.

Abstract

The Handwritten Text Recognition problem has been a challenge for researchers for the last few decades, especially in the domain of computer vision, a subdomain of pattern recognition. Variability of texts amongst writers, cursiveness, and different font styles of handwritten texts with degradation of historical text images make it a challenging problem. Recognizing scanned document images in neural network-based systems typically involves a two-step approach: segmentation and recognition. However, this method has several drawbacks. These shortcomings encompass challenges in identifying text regions, analyzing layout diversity within pages, and establishing accurate ground truth segmentation. Consequently, these processes are prone to errors, leading to bottlenecks in achieving high recognition accuracies. Thus, in this study, we present an end-to-end paragraph recognition system that incorporates internal line segmentation and gated convolutional layers based encoder. The gating is a mechanism that controls the flow of information and allows to adaptively selection of the more relevant features in handwritten text recognition models. The attention module plays an important role in performing internal line segmentation, allowing the page to be processed line-by-line. During the decoding step, we have integrated a connectionist temporal classification-based word beam search decoder as a post-processing step. In this work, we have extended existing LexiconNet by carefully applying and utilizing gated convolutional layers in the existing deep neural network. Our results at line and page levels also favour our new GatedLexiconNet. This study reported character error rates of 2.27% on IAM, 0.9% on RIMES, and 2.13% on READ-16, and word error rates of 5.73% on IAM, 2.76% on RIMES, and 6.52% on READ-2016 datasets.

GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System

TL;DR

GatedLexiconNet addresses offline handwritten paragraph recognition with an end-to-end, segmentation-free approach. It integrates a gated convolutional encoder, vertical attention for internal line segmentation, parcel-end detection, and a lexicon-driven Word Beam Search decoder to produce line-by-line transcriptions from paragraph images. The model demonstrates notable improvements in character and word error rates on IAM, RIMES, and READ-2016 compared to segmentation-based baselines, validating the effectiveness of gating and lexicon-aware decoding for complex handwriting layouts. Although effective, the approach currently assumes single-column layouts and incurs substantial computation, guiding future work toward scalability and multi-column handling.

Abstract

The Handwritten Text Recognition problem has been a challenge for researchers for the last few decades, especially in the domain of computer vision, a subdomain of pattern recognition. Variability of texts amongst writers, cursiveness, and different font styles of handwritten texts with degradation of historical text images make it a challenging problem. Recognizing scanned document images in neural network-based systems typically involves a two-step approach: segmentation and recognition. However, this method has several drawbacks. These shortcomings encompass challenges in identifying text regions, analyzing layout diversity within pages, and establishing accurate ground truth segmentation. Consequently, these processes are prone to errors, leading to bottlenecks in achieving high recognition accuracies. Thus, in this study, we present an end-to-end paragraph recognition system that incorporates internal line segmentation and gated convolutional layers based encoder. The gating is a mechanism that controls the flow of information and allows to adaptively selection of the more relevant features in handwritten text recognition models. The attention module plays an important role in performing internal line segmentation, allowing the page to be processed line-by-line. During the decoding step, we have integrated a connectionist temporal classification-based word beam search decoder as a post-processing step. In this work, we have extended existing LexiconNet by carefully applying and utilizing gated convolutional layers in the existing deep neural network. Our results at line and page levels also favour our new GatedLexiconNet. This study reported character error rates of 2.27% on IAM, 0.9% on RIMES, and 2.13% on READ-16, and word error rates of 5.73% on IAM, 2.76% on RIMES, and 6.52% on READ-2016 datasets.
Paper Structure (29 sections, 7 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 29 sections, 7 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Overview
  4. Encoder Architecture
  5. Convolutional Layers
  6. Gated Convolutional Layer
  7. Instance Normalization
  8. Depthwise Separable Convolutional Layers
  9. Residual Connections
  10. Attention
  11. Internal Line Segmentation
  12. Paragraph End Detection
  13. Decoder
  14. LSTM and Convolution-based Decoder
  15. WBS Decoder
  16. ...and 14 more sections

Figures (4)

  • Figure 1: System Overview of GatedLexionNet
  • Figure 2: Convolutional block architecture of present work compared with VAN
  • Figure 4: Comparison graph of present work with other state-of-the-art line level works on READ-2016 Dataset
  • Figure 5: Comparison graph of present work with other state-of-the-art line level works on RIMES Dataset