LineCounter: Learning Handwritten Text Line Segmentation by Counting

TL;DR

The paper tackles handwritten text line segmentation (HTLS) by shifting from traditional semantic segmentation or object-detection formulations to a Line Counting approach that predicts per-pixel line indices from the top, enforcing monotonicity to reflect line order. It introduces LineCounter, an encoder-counter-decoder network with horizontal and vertical GRU-based propagation and a cumsum-based monotone activation to ensure ordered line numbers. Experiments on ICDAR2013-HSC, HIT-MW, and VML-AHTE show LineCounter achieves state-of-the-art F-measure with around 2.5 FPS on ~13M-parameter models, outperforming existing methods without heavy post-processing. The work demonstrates robust cross-script HTLS capabilities (Latin/Bangla, Chinese, Arabic) and provides public code for end-to-end HTLS with improved efficiency and accuracy.

Abstract

Handwritten Text Line Segmentation (HTLS) is a low-level but important task for many higher-level document processing tasks like handwritten text recognition. It is often formulated in terms of semantic segmentation or object detection in deep learning. However, both formulations have serious shortcomings. The former requires heavy post-processing of splitting/merging adjacent segments, while the latter may fail on dense or curved texts. In this paper, we propose a novel Line Counting formulation for HTLS -- that involves counting the number of text lines from the top at every pixel location. This formulation helps learn an end-to-end HTLS solution that directly predicts per-pixel line number for a given document image. Furthermore, we propose a deep neural network (DNN) model LineCounter to perform HTLS through the Line Counting formulation. Our extensive experiments on the three public datasets (ICDAR2013-HSC, HIT-MW, and VML-AHTE) demonstrate that LineCounter outperforms state-of-the-art HTLS approaches. Source code is available at https://github.com/Leedeng/Line-Counter.

Figures (3)

• Figure 1: Sample LineCounter input document images (upper) and output per-pixel line counting maps (lower). Samples from left to right are from the ICDAR2013-HSC 2013icdar, HIT-MW su2007corpus, and VML-AHTE dataset barakat2020unsupervised, respectively. Predicted line numbers are rounded to nearest integers and color-coded according to the rightmost color scheme. Texts are artificially superimposed on predictions for better visualization. Best viewed digitally in color and zoomed-in.
• Figure 2: The proposed LineCounter network architecture. Each Conv2D block with blue border represents a module of Conv2D + BatchNorm + ReLU, convolution parameters are of format ${filters}@${ksize}$\times$${ksize}, and a darker block indicates more filters. Arrows denote down-sampling ($\downarrow$) or up-sampling ($\uparrow$). GRU and GRU apply the bidirectional recurrent analysis by treating the time axis as the column and row axes, respectively. They both use hard_sigmoid recurrent activation, and tanh output activation. Conv2D block with red border is a convolutional layer with novel cumsum activation introduced in Sec. \ref{['sec.counter']}. The output line number map is rounded and color coded according to the rightmost coloring scheme, and texts on the map are artificially superimposed for better visualization.
• Figure 3: Sample LineCounter outputs w/o and w/ the cumsum activation. The output line number map is rounded and color-coded according to the rightmost coloring scheme, and texts on the map are artificially superimposed for better visualization.