An N-Point Linear Solver for Line and Motion Estimation with Event Cameras

TL;DR

The paper introduces a linear N-point solver for line- and motion-estimation with event cameras by adopting a minimal angle-axis line representation, enabling closed-form recovery of line parameters and partial velocities from as few as five events. It replaces prior polynomial solvers with a fast linear system, provides explicit handling of degeneracies and multiple solutions, and introduces a velocity-averaging scheme to fuse observations from multiple lines into a full velocity estimate. Integrated with GC-RANSAC and a geometrically motivated residual, the approach yields substantial speedups (over 600x) and improved numerical stability in both synthetic and real-world experiments. The work advances real-time, line-based motion estimation with events and offers a framework compatible with standard frame-based line features as well as potential extensions with IMU data and uncertainties.

Abstract

Event cameras respond primarily to edges--formed by strong gradients--and are thus particularly well-suited for line-based motion estimation. Recent work has shown that events generated by a single line each satisfy a polynomial constraint which describes a manifold in the space-time volume. Multiple such constraints can be solved simultaneously to recover the partial linear velocity and line parameters. In this work, we show that, with a suitable line parametrization, this system of constraints is actually linear in the unknowns, which allows us to design a novel linear solver. Unlike existing solvers, our linear solver (i) is fast and numerically stable since it does not rely on expensive root finding, (ii) can solve both minimal and overdetermined systems with more than 5 events, and (iii) admits the characterization of all degenerate cases and multiple solutions. The found line parameters are singularity-free and have a fixed scale, which eliminates the need for auxiliary constraints typically encountered in previous work. To recover the full linear camera velocity we fuse observations from multiple lines with a novel velocity averaging scheme that relies on a geometrically-motivated residual, and thus solves the problem more efficiently than previous schemes which minimize an algebraic residual. Extensive experiments in synthetic and real-world settings demonstrate that our method surpasses the previous work in numerical stability, and operates over 600 times faster.

Figures (7)

• Figure 1: Incidence relationship between the line $\mathbf{L}$, and the bearing vector $\mathbf{f}_{j}'$ of event. We parameterize the line with the rotation matrix $\mathbf{R}_{\ell}=[\mathbf{e}^{\ell}_{1} \, \mathbf{e}^{\ell}_{2} \, \mathbf{e}^{\ell}_{3}]$. Since scale is unobservable, we select the point $-\mathbf{e}^{\ell}_{3}$ at unit depth to lie on the line, and $\mathbf{e}^{\ell}_{1}$ to indicate its direction. Due to the aperture problem, we can only observe the projected camera velocity $\hat{\mathbf{v}}$ with components $u^{\ell}_{y}$ and $u^{\ell}_{z}$ in the $\mathbf{e}^{\ell}_{2}$ and $\mathbf{e}^{\ell}_{3}$ direction respectively.
• Figure 2: Multiplicity of solutions to the incidence relation in Eq. \ref{['eq:incidence_final']}. While $S_0$ and $S_3$ have the line in front of the camera, $S_1$ and $S_2$ have the line behind the camera. The solution pairs $S_0, S_3$ and $S_1, S_2$ differ in the orientation of $\mathbf{e}^{\ell}_{1}$, which comes from the ambiguity of defining the line direction. In solutions with the line behind the camera, the measured projected camera velocity, $\mathbf{R}_\ell \mathbf{u}_\ell$ returned by the solver is the negative of the true projected velocity $\hat{\mathbf{v}}$. However, these solutions can be discarded by enforcing the condition in Eq. \ref{['eq:solution_checking']}.
• Figure 3: (i) The line constraint in Eq. \ref{['eq:line_constraint']} dictates that the 90-degree rotated measured velocity $\mathbf{R}_{{\ell}i} \mathbf{R}_{\frac{\pi}{2}}\mathbf{u}_{{\ell}i}$ should be perpendicular to $\hat{\mathbf{v}}_{i}$, or $\mathbf{v}^\intercal \mathbf{H}_i^\intercal \mathbf{R}_{{\ell}i} \mathbf{R}_{\frac{\pi}{2}}\mathbf{u}_{{\ell}i} = 0$. (ii) Each such constraint spans a two-dimensional subspace, in which $\mathbf{v}$ must reside. With a minimum of two such subspaces, the velocity can be found.
• Figure 4: Condensed analysis of the number of used events (left) and used lines (right) over three types of representative noise.
• Figure 5: Analysis of the number of used events over three types of representative noise, i.e. pixel noise, timestamp jitter, and gyroscope noise.