Parallax-Tolerant Image Stitching with Epipolar Displacement Field

TL;DR

The paper tackles parallax-induced misalignments in image stitching by introducing a parallax-tolerant framework grounded in epipolar geometry. It combines plane-induced and infinite homographies with a thin-plate-spline–based epipolar displacement field to warp images along epipolar lines, preserving global projection while aligning overlaps. Efficiency is achieved via a grid-based TPS approach to compute the displacement field, enabling scalable stitching across large baselines. Empirical results show competitive qualitative and quantitative performance against state-of-the-art methods, with improved global projectivity and reduced artifacts in non-overlapping regions.

Abstract

Image stitching with parallax is still a challenging task. Existing methods often struggle to maintain both the local and global structures of the image while reducing alignment artifacts and warping distortions. In this paper, we propose a novel approach that utilizes epipolar geometry to establish a warping technique based on the epipolar displacement field. Initially, the warping rule for pixels in the epipolar geometry is established through the infinite homography. Subsequently, the epipolar displacement field, which represents the sliding distance of the warped pixel along the epipolar line, is formulated by thin-plate splines based on the principle of local elastic deformation. The stitching result can be generated by inversely warping the pixels according to the epipolar displacement field. This method incorporates the epipolar constraints in the warping rule, which ensures high-quality alignment and maintains the projectivity of the panorama. Qualitative and quantitative comparative experiments demonstrate the competitiveness of the proposed method for stitching images with large parallax.

Figures (8)

• Figure 1: Cases that fail to meet the epipolar constraint. Three epipolar lines, highlighted in yellow, along with several corresponding point sets, are identified in the upper half of the target image. However, the points that are transformed into the panoramic image using existing stitching methods no longer adhere to the epipolar lines as prescribed by the warping rule. This occurs mainly in the non-overlapping areas.
• Figure 2: Warp image with the Epipolar Displacement Field. The transformation based on the infinite homography is equivalent to shifting the viewpoint of the object along the baseline direction of the stereo rig towards the reference viewpoint, forming a concentric projection model. In the image plane of the reference viewpoint, the warped image is formed by the intersection of the field of view from the new viewpoint and the reference image plane. In the stitching viewpoint, both $\textbf{x}_\infty$ and $\textbf{x}'$ lie on the epipolar line to maintain the epipolar constraint, while their distances give rise to the epipolar displacement field. With the help of this field, image warping aligns the pixels to the exact position along the epipolar lines.
• Figure 3: Epipolar constraints preservation: points on the epipolar line in the target image persist on the corresponding epipolar line after warping.
• Figure 4: Comparison of stitching quality on a work table scene. From top to bottom are the results using APAP, SPW, LPC, UDIS2, REW and the method proposed in this paper. Regions are extracted and enlarged to show the alignment performance of the different methods in this challenging scenario.
• Figure 5: More Comparisons of the image stitching results obtained by the proposed method with APAP, SPW, LPC, UDIS2 and REW. The test image pairs are derived from the open source code of the literature jia2021leveragingzhang2014parallax.