Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model

Ziqi Xie, Weidong Zhao, Xianhui Liu, Jian Zhao, Ning Jia

TL;DR

SRStitcher reformulates fusion and rectangling as a single, training-free inpainting task guided by weighted masks, enabling a robust, single-inference stitching pipeline despite registration errors. The method builds a coarse fusion image $I_{CF}$, defines seam and content masks, and solves a unified inpainting problem through a diffusion-based reverse process with a weighted mask strategy (WMGRP). Evaluations on the large UDIS-D dataset show superior quantitative metrics (HIQA, CLIPIQA, CCS) and qualitative results, with strong generalization and interpretability demonstrated across variants and user studies. The work underscores the power of large-scale generative priors to simplify complex image-composition pipelines and points to future directions that integrate registration more tightly within the unified framework.

Abstract

Deep learning-based image stitching pipelines are typically divided into three cascading stages: registration, fusion, and rectangling. Each stage requires its own network training and is tightly coupled to the others, leading to error propagation and posing significant challenges to parameter tuning and system stability. This paper proposes the Simple and Robust Stitcher (SRStitcher), which revolutionizes the image stitching pipeline by simplifying the fusion and rectangling stages into a unified inpainting model, requiring no model training or fine-tuning. We reformulate the problem definitions of the fusion and rectangling stages and demonstrate that they can be effectively integrated into an inpainting task. Furthermore, we design the weighted masks to guide the reverse process in a pre-trained largescale diffusion model, implementing this integrated inpainting task in a single inference. Through extensive experimentation, we verify the interpretability and generalization capabilities of this unified model, demonstrating that SRStitcher outperforms state-of-the-art methods in both performance and stability. Code: https://github.com/yayoyo66/SRStitcher

Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model

TL;DR

SRStitcher reformulates fusion and rectangling as a single, training-free inpainting task guided by weighted masks, enabling a robust, single-inference stitching pipeline despite registration errors. The method builds a coarse fusion image , defines seam and content masks, and solves a unified inpainting problem through a diffusion-based reverse process with a weighted mask strategy (WMGRP). Evaluations on the large UDIS-D dataset show superior quantitative metrics (HIQA, CLIPIQA, CCS) and qualitative results, with strong generalization and interpretability demonstrated across variants and user studies. The work underscores the power of large-scale generative priors to simplify complex image-composition pipelines and points to future directions that integrate registration more tightly within the unified framework.

Abstract

Deep learning-based image stitching pipelines are typically divided into three cascading stages: registration, fusion, and rectangling. Each stage requires its own network training and is tightly coupled to the others, leading to error propagation and posing significant challenges to parameter tuning and system stability. This paper proposes the Simple and Robust Stitcher (SRStitcher), which revolutionizes the image stitching pipeline by simplifying the fusion and rectangling stages into a unified inpainting model, requiring no model training or fine-tuning. We reformulate the problem definitions of the fusion and rectangling stages and demonstrate that they can be effectively integrated into an inpainting task. Furthermore, we design the weighted masks to guide the reverse process in a pre-trained largescale diffusion model, implementing this integrated inpainting task in a single inference. Through extensive experimentation, we verify the interpretability and generalization capabilities of this unified model, demonstrating that SRStitcher outperforms state-of-the-art methods in both performance and stability. Code: https://github.com/yayoyo66/SRStitcher
Paper Structure (34 sections, 12 equations, 15 figures, 2 tables, 1 algorithm)

This paper contains 34 sections, 12 equations, 15 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Methodology
  4. Unified inpainting model
  5. Weighted mask guided reverse process
  6. Experiments
  7. Experimental setup
  8. Quantitative evaluation
  9. Qualitative evaluation
  10. User study
  11. Ablation study
  12. Discussion and conclusion
  13. More details of SRStitcher
  14. More analysis of the designs described in the main paper
  15. Physical implications of weighted masks $\widetilde{M}_{init}(x,y)$ and $\widetilde{M}_{inpaint}(x,y)$
  16. ...and 19 more sections

Figures (15)

  • Figure 1: Comparison between existing pipeline and SRStitcher. Process ① is implemented by UDIS Nie_2021, process ② by UDIS++ nie2023parallax, process ③ by DeepRectangling (DR) nie2022deep, process ④ by Eq. \ref{['eq:initmask']} and Eq. \ref{['eq:inpaintmask']}. The corresponding partial images, I and IV, illustrate how SRStitcher effectively corrects the apparent misalignment of a pillar. Similarly, the partial images II and III demonstrate how SRStitcher repairs the blurry coarse rectangling areas. SRStitcher can be applied to both UDIS and UDIS++ aligned images and get similar stitched results.
  • Figure 2: Qualitative evaluation results. All visual results are obtained with seed $0$.
  • Figure 3: User study on visual quality. The results are averaged across 20 participants, with the percentage on the ordinate axis.
  • Figure 4: Ablation study results. CCS on each image is the average score of $UDIS-D_{test}$ with this hyper-parameter and seed $0$, not a single image.
  • Figure 5: The effects of each design on SRStitcher results. We demonstrate how each design element influences the results of SRStitcher by removing them one at a time and observing the changes.
  • ...and 10 more figures