Leveraging Contrastive Learning for a Similarity-Guided Tampered Document Data Generation Pipeline

TL;DR

This work tackles data scarcity in tampered-document detection by introducing two auxiliary networks: a crop-similarity network trained via contrastive learning and a bounding-box quality network, which together drive a unified data-generation pipeline capable of producing diverse, high-fidelity tampered document images across five manipulation types. The Syn2Real-TDoc data-generation framework leverages OCR-derived line segments and context-aware cropping, enabling on-the-fly or offline generation of a large synthetic dataset (TDoc-2.8M) that substantially improves zero-shot and fine-tuned detection and localization across RTM, FindIt, and FindItAgain benchmarks. Ablation studies confirm that both F_theta and G_theta contribute complementary benefits, with significant gains over rule-based baselines like DocTamper, and the approach generalizes to AI-generated tampering scenarios. The authors also release code, pretrained models, and the TDoc-2.8M dataset, highlighting practical impact for robust document forgery detection in real-world settings.

Abstract

Detecting tampered text in document images is a challenging task due to data scarcity. To address this, previous work has attempted to generate tampered documents using rule-based methods. However, the resulting documents often suffer from limited variety and poor visual quality, typically leaving highly visible artifacts that are rarely observed in real-world manipulations. This undermines the model's ability to learn robust, generalizable features and results in poor performance on real-world data. Motivated by this discrepancy, we propose a novel method for generating high-quality tampered document images. We first train an auxiliary network to compare text crops, leveraging contrastive learning with a novel strategy for defining positive pairs and their corresponding negatives. We also train a second auxiliary network to evaluate whether a crop tightly encloses the intended characters, without cutting off parts of characters or including parts of adjacent ones. Using a carefully designed generation pipeline that leverages both networks, we introduce a framework capable of producing diverse, high-quality tampered document images. We assess the effectiveness of our data generation pipeline by training multiple models on datasets derived from the same source images, generated using our method and existing approaches, under identical training protocols. Evaluating these models on various open-source datasets shows that our pipeline yields consistent performance improvements across architectures and datasets.

Figures (10)

• Figure 1: Illustration of some failure cases in tampered document generation caused by a visual mismatch between the source and target crops, resulting in obvious visual artifacts.
• Figure 2: Illustration of positive and negative pairs in our contrastive setting. The red box denotes the anchor, the green its positive, and the blue boxes the negatives.
• Figure 3: Similarity scores from $\mathcal{F}_\theta$ between a target crop (left) and candidate crops. The score below each crop indicates its similarity to the target crop and is scaled by a factor of 10.
• Figure 4: Illustration of failure cases in document tampering due to poor bounding boxes: (A) an ill‑defined source box cutting through characters; (B) an ill‑defined target box overlapping adjacent text.
• Figure 5: A simplified illustration of our tampered document generation process.