CSPR-Net: Self-supervised Curved Surface Projection Rectification Network for Geometric Distortion Correction in Non-planar Projections

TL;DR

CSPR-Net tackles geometric distortions from projecting onto curved surfaces by learning bijective forward and backward mappings between projector and camera spaces using dual coordinate-based MLPs. The method employs cycle-consistency and a gradient-based self-supervised loss to rectify distortions without ground-truth deformation fields, producing high-precision pre-warped images for seamless projection. Across synthetic simulations and physical experiments, CSPR-Net consistently outperforms a 3rd-degree polynomial baseline in SSIM, RMSE, and PSNR, demonstrating robust reconstruction of curved-surface content. This calibration-free framework enables flexible projection mapping in spatial AR and projection-mapping contexts, with potential extensions to radiometric correction and defocus restoration.

Abstract

Projecting images onto non-planar surfaces inevitably introduces geometric distortions that degrade visual quality. Traditional correction methods often require tedious manual calibration or structured light sequences to establish pixel-wise correspondences. In this paper, we develop the Curved Surface Projection Rectification Network (CSPR-Net), a self-supervised deep learning framework for automated distortion correction. Our approach employs dual coordinate-based neural networks to learn the bi-directional mapping between the projector and camera spaces. By enforcing a robust cycle-consistency constraint, CSPR-Net autonomously resolves complex geometric transformations without requiring ground-truth deformation fields. Furthermore, a gradient-based loss function is introduced to mitigate the impact of complex ambient light interference and accurately capture high-frequency geometric variations. Quantitative evaluations in physical experimental scenarios demonstrate that CSPR-Net achieves a 20.7% improvement in end-to-end fidelity (SSIM) and outperforms the polynomial baseline by 3.8% and 5.4% in forward and inverse mapping in terms of SSIM respectively, effectively generating high-precision pre-warped images for seamless projection.

Figures (4)

• Figure 1: Overview of the proposed CSPR-Net. (a) Detailed architecture of CSPR-Net. The model utilizes a 4-layer fully connected MLP with Layer Normalization and LeakyReLU activations to regress coordinate displacements. A differentiable Spatial Transformer is integrated for end-to-end image warping; (b) The self-supervised training pipeline. The pipeline enforces spatial consistency between the projector and camera domains through a composite loss function, comprising forward/backward photometric losses, cycle-consistency losses, and smoothness regularization.
• Figure 2: Simulation environment setup and comparative results. (a) The virtual ray-tracing configuration comprising a projector-camera pair and a cylindrical surface; (b)-(c) Comparison of inverse mapping performance, where the proposed CSPR-Net (c) successfully restores a rectilinear grid unlike the polynomial baseline (b) which retains residual distortions; (d)-(e) Pixel-wise error heatmaps relative to the ground truth, demonstrating that our method (e) significantly minimizes reconstruction errors at the boundaries compared to the baseline (d).
• Figure 3: Experimental validation and performance analysis of CSPR-Net on a physical curved surfaces. (a) Visualization of the experimental dataset. The data is organized by rows and columns: the first column displays three distinct source calibration patterns featuring spatially varying grid densities and color transitions while the second column presents the corresponding distorted images captured by the camera, which exhibit severe non-linear warping due to the physical surface geometry; (b) Convergence analysis of the self-supervised objective. The plot illustrates the stable descent of the composite loss function and its constituent terms, including forward/backward photometric, cycle-consistency, geometric regularization, and mask consistency losses over 6,000 training iterations; (c) Visual progression of the inverse geometric mapping during training. Reconstructed projector-space images from distorted camera inputs are shown at $Iter = 1000, 3000,$ and $6000$. As training proceeds, CSPR-Net progressively rectifies the curvilinear distortions to recover the underlying rectilinear grid structure; (d) Geometric correction results of the proposed CSPR-Net. This subfigure displays the generated pre-warped image and its corresponding physical display after being projected onto the curved surface, demonstrating high-fidelity rectification; (e) Geometric correction results of the polynomial fitting baseline. This subfigure shows the pre-warped image and the associated physical projection result generated by the 3rd-degree polynomial method. Compared to CSPR-Net, the baseline exhibits noticeable residual curvilinear distortion at the image boundaries, highlighting its limitations in accurately modeling complex non-linearities in regions with steep geometric gradients.
• Figure 4: Qualitative evaluation of CSPR-Net on three distinct curved surfaces using an unseen scenic test image. From top to bottom, the rows represent image sets captured on W, V, and M-shaped surfaces respectively. The columns represent the stages of the rectification process: the first column (a, d, g) displays the pre-warped images generated by our framework; the second column (b, e, h) shows the direct projection results without any geometric compensation; and the third column (c, f, i) captures the final rectified projections. The results demonstrate that CSPR-Net consistently achieves high-fidelity, rectilinear displays across diverse complex topographies.