LF-PGVIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras using Points and Geodesic Segments

TL;DR

An Omnidirectional Curve Segment Detection (OCSD) method combined with a camera model which is applicable to images with large distortions, such as panoramic annular images, fisheye images, and various panoramic images is proposed.

Abstract

In this paper, we propose LF-PGVIO, a Visual-Inertial-Odometry (VIO) framework for large Field-of-View (FoV) cameras with a negative plane using points and geodesic segments. The purpose of our research is to unleash the potential of point-line odometry with large-FoV omnidirectional cameras, even for cameras with negative-plane FoV. To achieve this, we propose an Omnidirectional Curve Segment Detection (OCSD) method combined with a camera model which is applicable to images with large distortions, such as panoramic annular images, fisheye images, and various panoramic images. The geodesic segment is sliced into multiple straight-line segments based on the radian and descriptors are extracted and recombined. Descriptor matching establishes the constraint relationship between 3D line segments in multiple frames. In our VIO system, line feature residual is also extended to support large-FoV cameras. Extensive evaluations on public datasets demonstrate the superior accuracy and robustness of LF-PGVIO compared to state-of-the-art methods. The source code will be made publicly available at https://github.com/flysoaryun/LF-PGVIO.

Figures (14)

• Figure 1: Three typical omnidirectional images that may have a negative plane and the corresponding FoV. (a) Fisheye FoV and Fisheye image. (b) Panoramic annular FoV and Panoramic annular image. (c) Panoramic FoV and Panoramic image.
• Figure 2: $\textbf{(a)}$ Our car experiment platform with a Panoramic Annular Lens (PAL) camera, a Livox-Mid-360 LiDAR, an IMU sensor, and an onboard computer. $\textbf{(b)}$ Top view of trajectories of different algorithms and ground truth for the OD01 sequence in outdoor experiments. The car platform stacks images in a residual manner with a $0.5s$ interval on the first frame, and the trajectory aligns with the ground truth.
• Figure 3: Overview of the proposed LF-PGVIO system. The sensor data includes RGB images and IMU. Our sliding window optimization approach consists of the following four parts: point feature residual, line feature residual, IMU residual, and marginalization residual.
• Figure 4: The projection from an orange 3D line onto a geodesic segment on a unit sphere, and a geodesic segment onto a curved segment on an image. The red dashed line is the great circle where the geodesic segment lies and the green vector represents one of the unit vectors that is perpendicular to the plane containing the great circle.
• Figure 5: The distance between a purple pixel and an orange curve segment in the image. The purple and red points in the image correspond to their counterparts on the unit sphere, which are also purple and red respectively. The orange curve segment on the image corresponds to the orange geodesic segment on the unit sphere. This orange geodesic segment lies on the great circle represented by the red dashed line.