RiemanLine: Riemannian Manifold Representation of 3D Lines for Factor Graph Optimization

TL;DR

This work tackles the challenge of accurately estimating camera poses and reconstructing 3D lines in environments with strong structural regularities. It introduces RiemanLine, a unified minimal representation that decouples each line into a shared vanishing direction on $S^2$ and a local scaled normal on $_\lambda S^1$, and extends this to parallel-line groups, yielding 2+2k degrees of freedom for k lines. The method is embedded in a manifold-based, co-visibility factor-graph optimization framework, combining point and line measurements with explicit parallelism constraints. Experiments on ICL-NUIM, TartanAir, and synthetic simulations show improved pose accuracy, reduced parameter count, and better convergence stability compared to traditional representations, especially in structurally rich indoor scenes.

Abstract

Minimal parametrization of 3D lines plays a critical role in camera localization and structural mapping. Existing representations in robotics and computer vision predominantly handle independent lines, overlooking structural regularities such as sets of parallel lines that are pervasive in man-made environments. This paper introduces \textbf{RiemanLine}, a unified minimal representation for 3D lines formulated on Riemannian manifolds that jointly accommodates both individual lines and parallel-line groups. Our key idea is to decouple each line landmark into global and local components: a shared vanishing direction optimized on the unit sphere $\mathcal{S}^2$, and scaled normal vectors constrained on orthogonal subspaces, enabling compact encoding of structural regularities. For $n$ parallel lines, the proposed representation reduces the parameter space from $4n$ (orthonormal form) to $2n+2$, naturally embedding parallelism without explicit constraints. We further integrate this parameterization into a factor graph framework, allowing global direction alignment and local reprojection optimization within a unified manifold-based bundle adjustment. Extensive experiments on ICL-NUIM, TartanAir, and synthetic benchmarks demonstrate that our method achieves significantly more accurate pose estimation and line reconstruction, while reducing parameter dimensionality and improving convergence stability.

Figures (7)

• Figure 1: An illustration of co-visibility factor graph optimization based points and lines. The initial factor graphs are depicted in the first row, with landmarks and trajectories colored light green and scarlet, respectively. The convergence curves for different representations are plotted in the second row. The optimized results based on the proposed Point_RiemanLine method and ground-truth graphs are presented in the last rows, respectively.
• Figure 2: Illustration of the proposed parametrization for a line landmark $\mathbf{\mathcal{L}}^j$. The vanishing direction vector $\mathbf{u}_2$ ($\|\mathbf{u}_2\|=1$) and scaled normal vector $\omega_{n}^j\mathbf{u}_1$ ( $\|\mathbf{u}_1\|=1, \omega_{n}^j>0$) are optimized on the tangent spaces $T_{\mathbf{u}_2}\mathcal{S}^2$ of the sphere and $T_{\mathbf{u}_1}{_\lambda}\mathcal{S}^{1}$ of the scaled circle, respectively.
• Figure 3: Illustration of the parametrization for two parallel line landmarks $\mathbf{\mathcal{L}}^i$ and $\mathbf{\mathcal{L}}^j$. The vanishing direction vector $\mathbf{u}_2$ and normalized normal vector $\mathbf{u}_1$ are optimized on the tangent spaces $T_{\mathbf{u}_2}\mathcal{S}^2$ of the sphere and $T_{\mathbf{u}_1}\mathcal{S}^{1}$ of the circle, respectively.
• Figure 4: Factor graph representations for different line-based structures. Left: conventional line re-projection factors. Right: the proposed parallel line representation, explicitly separating global and local components with re-projection factors.
• Figure 5: Line reconstruction errors of different methods in the Hospital sequence.