Visual Inertial Odometry using Focal Plane Binary Features (BIT-VIO)

TL;DR

BIT-VIO delivers a first-of-its-kind 6-DOF Visual Inertial Odometry that runs on a focal-plane sensor-processor (SCAMP-5) by fusing fast on-sensor BIT-VO features with a 400 Hz IMU via a loosely-coupled iterated EKF, achieving 300 FPS visual processing with low latency. The approach includes uncertainty propagation for focal-plane binary-edge features and extensive real-world validation against BIT-VO using ground-truth motion capture, showing smoother trajectories and reduced high-frequency noise. The results demonstrate the practicality of on-sensor vision processing for robust, low-latency VIO in resource-constrained mobile robotics, with plans to pursue tighter integration in future work.

Abstract

Focal-Plane Sensor-Processor Arrays (FPSP)s are an emerging technology that can execute vision algorithms directly on the image sensor. Unlike conventional cameras, FPSPs perform computation on the image plane -- at individual pixels -- enabling high frame rate image processing while consuming low power, making them ideal for mobile robotics. FPSPs, such as the SCAMP-5, use parallel processing and are based on the Single Instruction Multiple Data (SIMD) paradigm. In this paper, we present BIT-VIO, the first Visual Inertial Odometry (VIO) which utilises SCAMP-5.BIT-VIO is a loosely-coupled iterated Extended Kalman Filter (iEKF) which fuses together the visual odometry running fast at 300 FPS with predictions from 400 Hz IMU measurements to provide accurate and smooth trajectories.

Figures (6)

• Figure 1: Comparison of the proposed BIT-VIO algorithm and visual odometry (BIT-VO) overlaid on the reference ground-truth trajectory. BIT-VIO estimates are closer to the ground-truth trajectory compared to predictions from BIT-VO. Notice that BIT-VIO effectively removes the high-frequency noise visible in BIT-VO's trajectory. The plot was generated using evo grupp2017evo.
• Figure 2: Coordinate frame definition of the IMU and the SCAMP-5. In total, we define four coordinate frames. Notation $p_{A}^{B}$ and $q_{A}^{B}$ are used to represent transformation from $A$ to $B$.
• Figure 3: Pipeline of BIT-VIO. The multi-sensor fusion is to the left. BIT-VO is to the right. From the BIT-VO algorithm murai2020bit, the vision sensor utilizes the SCAMP-5 FPSP, highlighted in red. New corner/edge features are detected via the FPSP, off-putting computational load by allowing some image and signal processing to be done on the chip before transferring to a PC host or other external device to be further processed.
• Figure 4: FPSP SCAMP-5 reprojection error on a $256\times 256$ focal-plane imaging output with a less than one-pixel error (within the dashed black).
• Figure 5: Plots of the estimated translational RMSE (left) and rotational RMSE (middle) for Traj. G from Table \ref{['tb:table1']}. To the very right is the total translational RSME (top) and total rotational RMSE (bottom). For both translation and rotation, BIT-VIO is much closer and smoother to ground-truth data than IMU-alone and BIT-VO. The drift of the IMU-alone is very evident, as well as the high-frequency noise of BIT-VO.