DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks

TL;DR

DocRes presents a unified approach to document image restoration by combining a Dynamic Task-Specific Prompt (DTSPrompt) with a single restoration backbone to handle five tasks—dewarping, deshadowing, appearance enhancement, deblurring, and binarization. By extracting input-dependent priors as prompts and fusing them with the original image, DocRes achieves competitive results against task-specific models while supporting high-resolution inputs and efficient multitask training. Ablation studies show that DTSPrompt and cross-task synergy contribute to performance gains, and the method generalizes to out-of-domain data, validating the practicality of a unified restoration framework. The work suggests a promising direction toward foundation-model-like approaches for pixel-level document processing, with future work focused on richer priors and a trainable, shared prompt generator.

Abstract

Document image restoration is a crucial aspect of Document AI systems, as the quality of document images significantly influences the overall performance. Prevailing methods address distinct restoration tasks independently, leading to intricate systems and the incapability to harness the potential synergies of multi-task learning. To overcome this challenge, we propose DocRes, a generalist model that unifies five document image restoration tasks including dewarping, deshadowing, appearance enhancement, deblurring, and binarization. To instruct DocRes to perform various restoration tasks, we propose a novel visual prompt approach called Dynamic Task-Specific Prompt (DTSPrompt). The DTSPrompt for different tasks comprises distinct prior features, which are additional characteristics extracted from the input image. Beyond its role as a cue for task-specific execution, DTSPrompt can also serve as supplementary information to enhance the model's performance. Moreover, DTSPrompt is more flexible than prior visual prompt approaches as it can be seamlessly applied and adapted to inputs with high and variable resolutions. Experimental results demonstrate that DocRes achieves competitive or superior performance compared to existing state-of-the-art task-specific models. This underscores the potential of DocRes across a broader spectrum of document image restoration tasks. The source code is publicly available at https://github.com/ZZZHANG-jx/DocRes

Figures (9)

• Figure 1: DocRes is a generalist model that unifies five document image restoration tasks, including tasks of dewarping, deshadowing, appearance enhancement, deblurring, and binarization.
• Figure 1: More visualized results from DocRes. Zoom in for best view.
• Figure 2: The overall pipeline for DocRes. The document image to be restored, denoted as $I_s$, is initially fed into the DTSPrompt generator, which extracts specific prior features based on the task to form the DTSPrompt. Alongside $I_s$, DTSPrompt is input into the restoration network. It serves not only as a guidance for the restoration network on the particular task to be performed but also functions as auxiliary information derived from $I_s$ to improve performance.
• Figure 2: Failure case from DocRes when applying it for end-to-end camera-captured document image enhancement. Zoom in for best view.
• Figure 3: The DTSPrompt for different tasks is composed of distinct prior features. Most of these prior features are extracted from the input image, making them dynamic. Zoom in for the best view.