PLVS: A SLAM System with Points, Lines, Volumetric Mapping, and 3D Incremental Segmentation

TL;DR

PLVS presents a unified, CPU-based SLAM framework that integrates sparse feature tracking (points and lines), parallel volumetric mapping, and 3D incremental segmentation for RGB-D and stereo sensors. It extends ORB-SLAM with depth-informed line reprojection terms and supports multiple volumetric backends (octrees, Octomap, TSDF-based voxels/meshes) to yield consistent dense maps, even in the presence of loop closures. The approach combines a depth-aware line BA and an incremental segmentation module to produce robust, scalable reconstructions suitable for onboard robotics, with open-source implementation enabling replication and extension. The work demonstrates versatile mapping across diverse sensors and provides detailed mathematical tooling (appendices on line triangulation, derivatives, and covariances) to support accurate optimization and uncertainty handling. Overall, PLVS advances real-time, feature-rich SLAM with integrated 3D segmentation and dense volumetric mapping on CPU hardware.

Abstract

This document presents PLVS: a real-time system that leverages sparse SLAM, volumetric mapping, and 3D unsupervised incremental segmentation. PLVS stands for Points, Lines, Volumetric mapping, and Segmentation. It supports RGB-D and Stereo cameras, which may be optionally equipped with IMUs. The SLAM module is keyframe-based, and extracts and tracks sparse points and line segments as features. Volumetric mapping runs in parallel with respect to the SLAM front-end and generates a 3D reconstruction of the explored environment by fusing point clouds backprojected from keyframes. Different volumetric mapping methods are supported and integrated in PLVS. We use a novel reprojection error to bundle-adjust line segments. This error exploits available depth information to stabilize the position estimates of line segment endpoints. An incremental and geometric-based segmentation method is implemented and integrated for RGB-D cameras in the PLVS framework. We present qualitative and quantitative evaluations of the PLVS framework on some publicly available datasets. The appendix details the adopted stereo line triangulation method and provides a derivation of the Jacobians we used for line error terms. The software is available as open-source.

Figures (8)

• Figure 1: Three aspects of a 3D reconstruction (a) line segments (b) point cloud (c) segmented point cloud. For simplicity, we do not show point features. The volumetric map has been obtained by using an octree-based model with 1cm resolution.
• Figure 2: Details of the 3D reconstruction of Fig. \ref{['Fig:PLVSMain']} (a) line segments (b) normals (c) point cloud (d) segmented point cloud.
• Figure 3: The schema shows how three line segment observations $\{(\bm{p}_i,\bm{q}_i)\}_{i=1}^{3}$ of the same 3D line segment $l$ are backprojected onto the 3D line at different elements of the class ${\@fontswitch\mathcal{C}}_l$.
• Figure 4: The generic 3D line segment $l$, its obversation $(\bm{p}_i,\bm{q}_i)$ in the image $(I_k,D_k)$ and the map endpoints $(\bm{P},\bm{Q})$ with their camera projections $(\bm{p},\bm{q}) = (\bm{\pi}_k(\bm{P}),\bm{\pi}_k(\bm{Q}))$. The 2D reprojection error $e_{2D}$ in eq. (\ref{['Eq:2DReprojectionError']}) combines the depicted perpendicular point-line distances of $\bm{p}$ and $\bm{q}$ from the line $(\bm{p}_i,\bm{q}_i)$. On the other hand, the 3D backprojection error $e_{3D}$ in eq. (\ref{['Eq:3DBackprojectionError']}) combines (i) the 3D perpendicular point-line distances of $\bm{P}$ and $\bm{Q}$ from the 3D line $(\bm{\beta}_k(\bm{p}_i),\bm{\beta}_k(\bm{q}_i))$ and (ii) the 3D endpoint-endpoint distances $\|\bm{P}-\bm{\beta}_k(\bm{p}_i)\|$ and $\|\bm{Q}-\bm{\beta}_k(\bm{q}_i)\|$.
• Figure 5: Two 3D maps of line segments built for the same enviroment. Left: the map built without using the backprojection error. Right: map obtained by including the backprojection error in BA.