Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Event Collapse in Contrast Maximization Frameworks

Shintaro Shiba, Yoshimitsu Aoki, Guillermo Gallego

TL;DR

This work identifies and addresses the event-collapse failure mode in Contrast Maximization (CMax) for event-based vision by introducing two collapse metrics grounded in space-time deformation: the divergence of the event transformation flow and area-based deformation via the Jacobian determinant. These metrics are integrated as regularizers into the CMax objective, yielding an augmented objective $J(\boldsymbol{\theta}) = -G(\boldsymbol{\theta}) + \lambda R(\boldsymbol{\theta})$ that discourages collapse without harming well-posed warps. Experiments on MVSEC, DSEC, and ECD show substantial reductions in end-to-end optical-flow errors and improved IWE sharpness for collapse-enabled warps, while leaving well-posed warps unaffected, and a sensitivity analysis clarifies the trade-offs in regularization strength. The results demonstrate that the proposed regularizers provide a robust, data-efficient solution to event collapse, enabling CMax to handle broader warp models and paving the way for more reliable event-based motion estimation and segmentation. The work also lays a foundation for extending these metrics to more complex warp families via finite-difference approximations, broadening the applicability of CMax in dynamic scenes.

Abstract

Contrast maximization (CMax) is a framework that provides state-of-the-art results on several event-based computer vision tasks, such as ego-motion or optical flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. As prior works have largely ignored the issue or proposed workarounds, it is imperative to analyze this phenomenon in detail. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation based on differential geometry and physics. We experimentally show on publicly available datasets that the proposed metrics mitigate event collapse and do not harm well-posed warps. To the best of our knowledge, regularizers based on the proposed metrics are the only effective solution against event collapse in the experimental settings considered, compared with other methods. We hope that this work inspires further research to tackle more complex warp models.

Event Collapse in Contrast Maximization Frameworks

TL;DR

This work identifies and addresses the event-collapse failure mode in Contrast Maximization (CMax) for event-based vision by introducing two collapse metrics grounded in space-time deformation: the divergence of the event transformation flow and area-based deformation via the Jacobian determinant. These metrics are integrated as regularizers into the CMax objective, yielding an augmented objective that discourages collapse without harming well-posed warps. Experiments on MVSEC, DSEC, and ECD show substantial reductions in end-to-end optical-flow errors and improved IWE sharpness for collapse-enabled warps, while leaving well-posed warps unaffected, and a sensitivity analysis clarifies the trade-offs in regularization strength. The results demonstrate that the proposed regularizers provide a robust, data-efficient solution to event collapse, enabling CMax to handle broader warp models and paving the way for more reliable event-based motion estimation and segmentation. The work also lays a foundation for extending these metrics to more complex warp families via finite-difference approximations, broadening the applicability of CMax in dynamic scenes.

Abstract

Contrast maximization (CMax) is a framework that provides state-of-the-art results on several event-based computer vision tasks, such as ego-motion or optical flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. As prior works have largely ignored the issue or proposed workarounds, it is imperative to analyze this phenomenon in detail. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation based on differential geometry and physics. We experimentally show on publicly available datasets that the proposed metrics mitigate event collapse and do not harm well-posed warps. To the best of our knowledge, regularizers based on the proposed metrics are the only effective solution against event collapse in the experimental settings considered, compared with other methods. We hope that this work inspires further research to tackle more complex warp models.
Paper Structure (32 sections, 46 equations, 8 figures, 3 tables)

This paper contains 32 sections, 46 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contrast Maximization
  4. Event Collapse
  5. Method
  6. How Event Cameras Work
  7. Mathematical Description of the CMax Framework
  8. Simplest Example of Event Collapse: 1 DOF
  9. Proposed Regularizers
  10. Divergence of the Event Transformation Flow
  11. Area-Based Deformation of the Event Transformation
  12. Higher DOF Warp Models
  13. Feature Flow
  14. Rotational Motion
  15. Planar Motion
  16. ...and 17 more sections

Figures (8)

  • Figure S1: Event Collapse.Left: Landscape of the image variance loss as a function of the warp parameter $h_z$. Right: The IWEs at the different $h_z$ marked in the landspace. (A) Original events (identity warp), accumulated over a small $\Delta t$ (polarity is not used). (B) Image of warped events (IWE) showing event collapse due to maximization of the objective function. (C) Desired IWE solution using our proposed regularizer: sharper than (A) while avoiding event collapse (C).
  • Figure S2: Proposed modification of the contrast maximization (CMax) framework in Gallego18cvprGallego19cvpr to also account for the degree of regularity (collapsing behavior) of the warp. Events are colored in red/blue according to their polarity. Reprinted/adapted with permission from Ref. Gallego19cvpr, 2019, Gallego et al.
  • Figure S3: Point trajectories (streamlines) defined on $x-y-t$ image space by various warps. (a) Zoom in/out warp from image center (1 DOF). (b) Constant image velocity warp (2 DOF). (c) Rotational warp around $X$ axis (3 DOF).
  • Figure S4: Divergence of different vector fields, $\nabla\cdot \mathbf{v}= \partial_x \mathbf{v}_x + \partial_y \mathbf{v}_y$. From left to right: contraction ("sink", leading to event collapse), expansion ("source"), and incompressible fields. Image adapted from khanacademy.org
  • Figure S5: Area deformation of various warps. An area of $dA~\text{pix}^2$ at $(\mathbf{x}_k,t_k)$ and is warped to $t_\text{ref}$, giving an area $dA' = |\det(\mathtt{J}_k)| dA~\text{pix}^2$ at $(\mathbf{x}'_k,t_\text{ref})$, where $\mathtt{J}_k \equiv \mathtt{J}(e_k) \equiv \mathtt{J}(\mathbf{x}_k,t_k;\boldsymbol{\theta})$ (see \ref{['eq:defJacobian']}). From left to right, increasing area amplification factor $|\det(\mathtt{J})| \in [0, \infty)$.
  • ...and 3 more figures