A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework

TL;DR

A novel, computationally efficient regularizer based on geometric principles to mitigate event collapse is proposed and is hoped that this work opens the door for future applications that unlocks the advantages of event cameras.

Abstract

Event cameras are emerging vision sensors and their advantages are suitable for various applications such as autonomous robots. Contrast maximization (CMax), which provides state-of-the-art accuracy on motion estimation using events, may suffer from an overfitting problem called event collapse. Prior works are computationally expensive or cannot alleviate the overfitting, which undermines the benefits of the CMax framework. We propose a novel, computationally efficient regularizer based on geometric principles to mitigate event collapse. The experiments show that the proposed regularizer achieves state-of-the-art accuracy results, while its reduced computational complexity makes it two to four times faster than previous approaches. To the best of our knowledge, our regularizer is the only effective solution for event collapse without trading off runtime. We hope our work opens the door for future applications that unlocks the advantages of event cameras.

Figures (7)

• Figure 1: Sample application of event cameras. Top: The advantages of event cameras are beneficial for robotics applications, such as autonomous driving. Bottom: The proposed regularizer discourages event collapse (left), and reveals sharp edges of the scene in a computationally efficient manner (right). (Top image licensed from Stock Photo ID 1102269152).
• Figure 2: Method overview. The proposed regularizer (blue line) is based on geometric principles and solely relies on motion parameters $\boldsymbol{\theta}$, while previous approaches (dashed line) are built from warped events Shiba22sensors. Adapted with permission from Ref. Gallego19cvpr, 2019, Gallego et al.
• Figure 3: Rate of change of area deformation. The warp $\mathbf{W}$ defines point trajectories $\boldsymbol{\gamma}(t)=(\mathbf{x}(t),t)$ in the space-time image domain. We define the regularizer $\mathcal{R}$ based on differential area deformation along $\boldsymbol{\gamma}(t)$. The rate of change of area is given by the derivative of the Jacobian $\mathtt{J}_{t,t+\Delta t}$.
• Figure 4: Regularizer $\mathcal{R}$ for the 1-DOF warp, \ref{['eq:regularizer:oneDOF']}.
• Figure 5: Qualitative results. (a) Original events. (b)-(d) Results without regularization: 1-DOF motion results (MVSEC Zhu18ral and DSEC Gehrig21ral) are trapped in global optima of event collapse, as shown in the IWEs (b). The regularizers in such collapse cases (c)-(d) are very large compared with the well-posed warp cases (boxes_rot and dynamic_rot rows). (e) Results with the proposed regularizer: it mitigates collapse for MVSEC and DSEC scenes while it does not harm the ECD scenes. Best viewed in the electronic version.