RSL-BA: Rolling Shutter Line Bundle Adjustment

TL;DR

RSL-BA introduces the first line-based rolling shutter bundle adjustment by leveraging Plücker line parameterization to handle RS distortions on 3D lines. It develops a cubic RS line projection curve and three robust reprojection errors (perpendicular, horizontal, and tangent), combined into two optimization formulations that are efficiently solved via Levenberg–Marquardt. The method shows theoretical and empirical resistance to plane, 2-views pure translation, and X-Y pure translation degeneracies, including a newly identified case, while achieving accuracy and efficiency comparable to point-based RSBA methods. Extensive synthetic and real-data experiments validate robustness and competitiveness, highlighting the practical value of incorporating line features into RS imaging pipelines. This work lays the groundwork for line-based RS-SfM/RS-SLAM and hybrid point-line approaches in rolling shutter environments.

Abstract

The line is a prevalent element in man-made environments, inherently encoding spatial structural information, thus making it a more robust choice for feature representation in practical applications. Despite its apparent advantages, previous rolling shutter bundle adjustment (RSBA) methods have only supported sparse feature points, which lack robustness, particularly in degenerate environments. In this paper, we introduce the first rolling shutter line-based bundle adjustment solution, RSL-BA. Specifically, we initially establish the rolling shutter camera line projection theory utilizing Plücker line parameterization. Subsequently, we derive a series of reprojection error formulations which are stable and efficient. Finally, we theoretically and experimentally demonstrate that our method can prevent three common degeneracies, one of which is first discovered in this paper. Extensive synthetic and real data experiments demonstrate that our method achieves efficiency and accuracy comparable to existing point-based rolling shutter bundle adjustment solutions.

Figures (8)

• Figure 1: The pipeline of proposed RSL-BA with some example results. Starting with the input of some RS images (left), the optimization of camera poses and 3D line coordinates is performed using proposed RSL-BA (middle), yielding well-reconstructed 3D line segments and accurately estimated camera poses (right).
• Figure 2: The illustration of proposed line-based reprojection errors given the spatial line ${L}$ and observed curve $\boldsymbol{\iota}$. Instead of directly measuring the distance between the observed curve and the projected curve, we split each curve into multiple points $q$, and aggregate the distance errors. Specifically, we first utilize the virtual projected line $l$ when observing it, as shown in (a). To measure the distance between each point $q$ and the corresponding virtual lines, we can use the perpendicular distance (b), the horizontal distance (c,d) or the tangent distance (e) as our basic metrics.
• Figure 3: The illustration of degeneracy resistance of our proposed method in 2-views pure translation degeneracy. (a) Degeneration in single-view scenarios. (b) Suppression of degeneration using line features in dual-view scenarios. (c) Inability of line features to suppress degeneration in special dual-view scenarios
• Figure 4: In the case of X-Y pure translation, optimization compresses points into a straight line. In a single view, pixel points parallel to the motion trajectory back-project to the same 3D point ((a) and (b)). (c) Different intersection points in space are equidistant from the camera plane. (d) All intersection points formed by back-projection constitute a straight line.
• Figure 5: Comparison of degenerate resistance ability between NMRSBA Albl2016 and proposed RSL-BA. Ground truth camera poses and structure are colored with black. This example illustrates that our proposed RSL-BA has the resistance ability against the plane and X-Y pure translation degeneracy.