SemanticStitch: Enhancing Image Coherence through Foreground-Aware Seam Carving

TL;DR

SemanticStitch addresses image stitching misalignment by incorporating semantic priors of foreground objects into seam placement. The method introduces a three-component pipeline with a Transformer-based salient-object detector, an object-aware seam identification module, and a dynamic, area-driven mask optimization alongside a composite loss $\ abla\mathcal{L}_{total}=\mathcal{L}_{comp}+\mathcal{L}_{excl}+\mathcal{L}_{smooth}$. Key contributions include a novel object-completeness loss, an exclusivity constraint, and a smoothness term, plus two real-world datasets for evaluating foreground integrity in stitched images. Experimental results on UDIS-D, DAVISProcessed10, and RealWorld400 demonstrate substantial improvements over traditional and deep-learning baselines, supported by ablation and user studies, underscoring practical applicability in panoramic and dynamic-scene stitching. The work advances semantic-aware stitching by reliably preserving foreground objects while delivering coherent seams and natural transitions.

Abstract

Image stitching often faces challenges due to varying capture angles, positional differences, and object movements, leading to misalignments and visual discrepancies. Traditional seam carving methods neglect semantic information, causing disruptions in foreground continuity. We introduce SemanticStitch, a deep learning-based framework that incorporates semantic priors of foreground objects to preserve their integrity and enhance visual coherence. Our approach includes a novel loss function that emphasizes the semantic integrity of salient objects, significantly improving stitching quality. We also present two specialized real-world datasets to evaluate our method's effectiveness. Experimental results demonstrate substantial improvements over traditional techniques, providing robust support for practical applications.

Figures (11)

• Figure 1: Comparison of traditional methods (e.g., Graph Cut) Fair comparisons across different datasets and methods are provided in subsequent sections.
• Figure 2: Comparison of image stitching methods. This figure illustrates the performance of our approach relative to other mainstream seam-based methods. The magnified views show that our method significantly outperforms others due to its object-aware design. Yellow arrows indicate foreground objects that are incorrectly truncated by the other methods. The two images in the bottom left corner of each method depict the seam carving process, which are then stitched together to produce the final result.
• Figure 3: Overview of the proposed method. The first component predicts a reliable warp between two images. The second component predicts the foreground object masks for both images. The third component determines a seam that preserves the integrity of the foreground objects during stitching.
• Figure 4: Symbol Definition: Symbols and corresponding illustrations of dynamic mask optimization.
• Figure 5: Detailed comparative results : Comparison of image stitching results using Dynamic Programming, Graph Cut, UDIS++, Voronoi, and our proposed method.