DELINE8K: A Synthetic Data Pipeline for the Semantic Segmentation of Historical Documents

TL;DR

This work tackles the challenge of robust multiclass semantic segmentation for historical documents, where existing synthetic datasets fall short in background diversity and class richness. The authors introduce DELINE8K, an 8,000-image, four-layer synthetic dataset (background, handwriting, printed text, form elements) designed to align with the NAFSS benchmark, and they generate diverse backgrounds using DALL·E alongside extensive cursive, print, and form content from multiple sources. A UNet with a ResNet50 encoder trains on the DELINE8K pipeline, with data augmentations and specialized loss (Dice) to handle class imbalance, and it demonstrates strong performance on NAFSS—outperforming models trained on prior datasets and benefiting notably from synthetic backgrounds (as shown in an ablation study). The work also provides a practical data synthesis framework, including multiple auxiliary datasets (DIBCO, SignaTR6K, NAF) and modular components, enabling targeted synthetic data for various historical-document segmentation tasks, with potential implications for improving OCR and document editing workflows on degraded archival materials.

Abstract

Document semantic segmentation is a promising avenue that can facilitate document analysis tasks, including optical character recognition (OCR), form classification, and document editing. Although several synthetic datasets have been developed to distinguish handwriting from printed text, they fall short in class variety and document diversity. We demonstrate the limitations of training on existing datasets when solving the National Archives Form Semantic Segmentation dataset (NAFSS), a dataset which we introduce. To address these limitations, we propose the most comprehensive document semantic segmentation synthesis pipeline to date, incorporating preprinted text, handwriting, and document backgrounds from over 10 sources to create the Document Element Layer INtegration Ensemble 8K, or DELINE8K dataset. Our customized dataset exhibits superior performance on the NAFSS benchmark, demonstrating it as a promising tool in further research. The DELINE8K dataset is available at https://github.com/Tahlor/deline8k.

Figures (7)

• Figure 1: A model trained on DIBCO exhibits pockets of success on the task of distinguishing text from handwriting. Insufficient examples where both text and handwriting are present in the image may contribute to the missed classifications.
• Figure 2: Evaluation on randomly selected patches of the NAF dataset, excluding blanks
• Figure 3: Synthetic data can be easily adapted to match a target document collection.
• Figure 4: A postcard presents a particular challenge due to the various fonts used and presences of stamps.
• Figure 5: Nonstandard print orientations should be considered when creating synthetic data.