LECalib: Line-Based Event Camera Calibration

TL;DR

Camera calibration for event-based vision is challenged by reliance on flashing patterns, grayscale image reconstruction, or frame-like features, which is inefficient in dynamic settings. We present LECalib, a line-based calibration framework that detects lines directly from asynchronous events, constructs an initial projection model from 2D-3D line correspondences, and refines parameters with non-linear optimization on the projection matrix $\mathbf{M}$ and its decomposition into intrinsic $\mathbf{K}$ and extrinsic $ (\mathbf{R},\mathbf{T}) $. The method supports both planar and non-planar lines and eliminates the need for specialized calibration boards or image reconstruction. Experiments on simulated and real data show accurate monocular and stereo calibration, robustness to distortion and noise, and broad applicability to typical man-made environments.

Abstract

Camera calibration is an essential prerequisite for event-based vision applications. Current event camera calibration methods typically involve using flashing patterns, reconstructing intensity images, and utilizing the features extracted from events. Existing methods are generally time-consuming and require manually placed calibration objects, which cannot meet the needs of rapidly changing scenarios. In this paper, we propose a line-based event camera calibration framework exploiting the geometric lines of commonly-encountered objects in man-made environments, e.g., doors, windows, boxes, etc. Different from previous methods, our method detects lines directly from event streams and leverages an event-line calibration model to generate the initial guess of camera parameters, which is suitable for both planar and non-planar lines. Then, a non-linear optimization is adopted to refine camera parameters. Both simulation and real-world experiments have demonstrated the feasibility and accuracy of our method, with validation performed on monocular and stereo event cameras. The source code is released at https://github.com/Zibin6/line_based_event_camera_calib.

Figures (6)

• Figure 1: Overview of line-based event camera calibration framework. Lines are extracted from the events and used for calibration. The linear solution is adopted to provide initial camera parameters, using 2D-3D line correspondences. Then, the non-linear optimization is utilized to refine the parameters by minimizing line-line distance.
• Figure 2: Illustration of the 3D line projection and the relative motion. ${\mathbf P_{i}}{\mathbf Q_{i}},{\mathbf P_{i+1}}{\mathbf Q_{i+1}}$ are detected lines at time ${{t}_{i}},{{t}_{i+1}}$, which are parameterized by its two endpoints. The relative motion of the event camera from time ${{t}_{i}}$ to time ${{t}_{i+1}}$ can be described by rotation $\Delta \mathbf R$ and translation $\Delta \mathbf T$.
• Figure 3: Line detection results of the simulated event cluster with the noise $\sigma =1$ pixels. In the third row, we present the simulated event clusters with and without distortion. The fourth row shows the results of the LSD method, where the event cluster is accumulated into a 2D image. The fifth row displays the results of our method. The extracted lines at the beginning of the cluster are shown in blue, and the ends in yellow.
• Figure 4: Simulation experiment results. The red and blue polylines correspond to the left axis, the black and purple polylines correspond to the right axis. (a) Median error with respect to varying image noise $\sigma$ from 0 to 5 pixels with fixed number of lines $n=25$, the focal length ${{f}_{x}}={{f}_{y}}=400$, and the principal point at the image center. (b) Median error with respect to varying focal length $f$ from 400 to 2000 pixels with fixed number of lines $n=25$, noise $\sigma=1$ pixel, and the principal point at the image center. (c) Median error with respect to varying number of lines $n$ from 6 to 30 with fixed image noise $\sigma=1$ pixel, the focal length ${{f}_{x}}={{f}_{y}}=400$, and the principal point at the image center. (d) Median error with respect to varying distortion coefficients $k_1$, $k_2$ from -0.1 to -0.5 with a step size of 0.1, with fixed image noise $\sigma=1$ pixel, the focal length ${{f}_{x}}={{f}_{y}}=400$, the number of lines $n=25$, and the principal point at the image center.
• Figure 5: Line extraction results of the real data. The second column displays the color image captured by DAVIS34, where lens distortion is observed at the edges, as marked in the red circle area. The third column displays the event clusters, which have already been accumulated onto the event plane. The fourth column displays the line detection results using the LSD method, marked in red. The fifth column shows the line detection results obtained by our method. The extracted lines at the beginning of the cluster are shown in blue, and the ends in yellow.