Line Drawing Guided Progressive Inpainting of Mural Damage

TL;DR

This work tackles mural image inpainting, a task challenged by large, irregular damages and color bias, by introducing a line-drawing guided progressive framework. The method splits the task into structure reconstruction (SRN) and color correction (CCN), with line drawings guiding the first stage and non-local attention aiding color harmony in the second. A dedicated DhMurals1714 dataset of 1,714 murals from Dunhuang Mogao Grottoes with corresponding line drawings is released to support research. Through extensive experiments against state-of-the-art baselines and real-mural cases, the approach achieves superior structural fidelity and color coherence, validated by qualitative, quantitative, and user-study results. The work advances heritage preservation by providing a practical, efficient, and line-guided solution for digital mural restoration, along with a publicly available dataset and code.

Abstract

Mural image inpainting is far less explored compared to its natural image counterpart and remains largely unsolved. Most existing image-inpainting methods tend to take the target image as the only input and directly repair the damage to generate a visually plausible result. These methods obtain high performance in restoration or completion of some pre-defined objects, e.g., human face, fabric texture, and printed texts, etc., however, are not suitable for repairing murals with varying subjects and large damaged areas. Moreover, due to discrete colors in paints, mural inpainting may suffer from apparent color bias. To this end, in this paper, we propose a line drawing guided progressive mural inpainting method. It divides the inpainting process into two steps: structure reconstruction and color correction, implemented by a structure reconstruction network (SRN) and a color correction network (CCN), respectively. In structure reconstruction, SRN utilizes the line drawing as an assistant to achieve large-scale content authenticity and structural stability. In color correction, CCN operates a local color adjustment for missing pixels which reduces the negative effects of color bias and edge jumping. The proposed approach is evaluated against the current state-of-the-art image inpainting methods. Qualitative and quantitative results demonstrate the superiority of the proposed method in mural image inpainting. The codes and data are available at https://github.com/qinnzou/mural-image-inpainting.

Figures (9)

• Figure 1: Inpainting with and without the assistance of line drawings. (a) An image of damaged mural. (b) Masks on the mural image. (c) Corresponding line drawings of the mural. (d) The inpainting result obtained without line drawings. (e) The inpainting result obtained with line drawings.
• Figure 2: The problem of color bias in mural inpainting. Top row: mural images with masks in white. Bottom row: the inpainting results. The color bias is apparent between the inside and outside of the mask.
• Figure 3: The system overview and network structure of the proposed network. The upper part illustrates the system overview, the lower part illustrates the details of network structure. The whole model is composed of a Structure Reconstruction Network (SRN) and a Color Correction Network (CCN). SRN generates an approximate result with a complete structure, which is input into the CCN to generate a color-consistent final result.
• Figure 4: Mural paintings and corresponding line drawings in the DhMurals1714 dataset.
• Figure 5: Inpainting results of six mural images obtained by our method and five comparison ones - Deepfillv2 2019FreeForm, StructureFlowren2019structureflow, ICTwan2021ict, EdgeConnect 2019EdgeConnect, and RFR li2020recurrent. The upper row showcases damaged murals with varying mask rates, specifically 10%, 20%, 30%, 40%, 50%, and 60%. The second row displays the corresponding line drawings, followed by six subsequent rows depicting the inpainting outcomes from each method. Columns 1, 2, and 4 represent the replicated murals, while columns 3, 5, and 6 depict real murals from the training set.