Binarizing Documents by Leveraging both Space and Frequency

TL;DR

Document binarization must separate ink from background under diverse degradations while leveraging both local stroke details and global page-level patterns. The authors introduce FourBi, a fully-convolutional U-Net-like network that uses Fast Fourier Convolutions to fuse local spatial information with global spectral context, trained on 256x256 patches and evaluated with larger patches at inference. A patch-overlap fusion post-processing strategy further mitigates border artifacts, and extensive experiments on DIBCO/H-DIBCO and non-Latin datasets show FourBi often surpasses Conv baselines and Transformer-based SotA methods while remaining parameter-efficient. This approach demonstrates that FFT-based global context can robustly handle varied degradations and scripts, offering a scalable alternative to ViT-based models in document analysis.

Abstract

Document Image Binarization is a well-known problem in Document Analysis and Computer Vision, although it is far from being solved. One of the main challenges of this task is that documents generally exhibit degradations and acquisition artifacts that can greatly vary throughout the page. Nonetheless, even when dealing with a local patch of the document, taking into account the overall appearance of a wide portion of the page can ease the prediction by enriching it with semantic information on the ink and background conditions. In this respect, approaches able to model both local and global information have been proven suitable for this task. In particular, recent applications of Vision Transformer (ViT)-based models, able to model short and long-range dependencies via the attention mechanism, have demonstrated their superiority over standard Convolution-based models, which instead struggle to model global dependencies. In this work, we propose an alternative solution based on the recently introduced Fast Fourier Convolutions, which overcomes the limitation of standard convolutions in modeling global information while requiring fewer parameters than ViTs. We validate the effectiveness of our approach via extensive experimental analysis considering different types of degradations.

Figures (6)

• Figure 1: Overview of our Document Binarization approach exploiting FFCs (FourBi). The architecture consists in a U-Net-like model in which the outputs of the down-scaling and the up-scaling blocks are combined via concatenation. The central part of the model features FFC Residual Blocks (FFC-RB). The FFC operator in these blocks combines local spatial information with global frequency information, which results in semantic-preserving precise binarization.
• Figure 2: We combine the predictions of our model on overlapping patches at each point based on the distance between the point of interest and the center of each patch so that the artifacts at the patches borders are reduced.
• Figure 3: FM (top-left), pFM (top-right), PSNR (bottom-left), and DRD (bottom-right) values obtained by our FFC-based model and a baseline variant exploiting only standard convolutions when tested on patches of different sizes from H-DIBCO18, with or without overlap.
• Figure 4: Results of the Conv-based baseline and our FFC-based model on $256{\times}256$ and $512{\times}512$ patches. Correct predictions are in green, errors in red.
• Figure 5: Average error distribution of our FFC-based model and the Conv-based baseline on multiple $256{\times}256$ and $512{\times}512$ patches from different datasets.