E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization

TL;DR

E-MoFlow tackles the ill-posed problem of simultaneously recovering optical flow and 6-DoF ego-motion from event-based data by introducing implicit regularization through continuous representations. Optical flow is modeled as a neural implicit field $\mathbf{u}_\theta(t,\mathbf{x})$, while ego-motion is captured with a cubic B-spline $\beta$, enforcing spatial-temporal smoothness without depth priors. Two differential losses—a contrast-maximization driven flow loss and a differential geometric loss derived from a differential form of the motion field equation—enable unsupervised, geometry-consistent optimization that circumvents local minima. The approach achieves state-of-the-art performance among unsupervised methods on MVSEC and competitive results against supervised baselines, demonstrating the efficacy of implicit regularization for event-based vision. Overall, E-MoFlow offers a robust, depth-free, unsupervised framework that unifies flow and motion estimation with strong practical implications for neuromorphic perception tasks.

Abstract

The estimation of optical flow and 6-DoF ego-motion, two fundamental tasks in 3D vision, has typically been addressed independently. For neuromorphic vision (e.g., event cameras), however, the lack of robust data association makes solving the two problems separately an ill-posed challenge, especially in the absence of supervision via ground truth. Existing works mitigate this ill-posedness by either enforcing the smoothness of the flow field via an explicit variational regularizer or leveraging explicit structure-and-motion priors in the parametrization to improve event alignment. The former notably introduces bias in results and computational overhead, while the latter, which parametrizes the optical flow in terms of the scene depth and the camera motion, often converges to suboptimal local minima. To address these issues, we propose an unsupervised framework that jointly optimizes egomotion and optical flow via implicit spatial-temporal and geometric regularization. First, by modeling camera's egomotion as a continuous spline and optical flow as an implicit neural representation, our method inherently embeds spatial-temporal coherence through inductive biases. Second, we incorporate structure-and-motion priors through differential geometric constraints, bypassing explicit depth estimation while maintaining rigorous geometric consistency. As a result, our framework (called E-MoFlow) unifies egomotion and optical flow estimation via implicit regularization under a fully unsupervised paradigm. Experiments demonstrate its versatility to general 6-DoF motion scenarios, achieving state-of-the-art performance among unsupervised methods and competitive even with supervised approaches.

Figures (12)

• Figure 1: Illustration of E-MoFlow.
• Figure 2: Training Pipeline of E-MoFlow. Given the input event data, we use differential flow loss Eq. \ref{['eq:cmax']} and differential geometric loss Eq. \ref{['eq:dec_6dof_motion']} to train the neural network Eq. \ref{['eq:neural_ode']} and spline parameters Eq. \ref{['eq:motion_spline']}. These two losses are optimized together until convergence. Solid arrows denote the forward process; dashed arrows denote gradient backpropagation.
• Figure 3: Landscape of different loss functions. By jointly estimating these two losses, we can avoid getting stuck in local minima, making it possible to solve the ill-posed problem.
• Figure 4: Qualitative results of 6-DoF motion estimation on the MVSEC dataset. Due to space constraints, this figure only presents the linear velocity estimation results. The angular velocity results are provided in the supplementary material. The top, middle, and bottom rows in each subfigure correspond to the $x$-axis, $y$-axis, and $z$-axis results, respectively.
• Figure 5: Qualitative results of optical flow estimation on the MVSEC dataset. The results in the first, second, and third rows correspond to sequences indoor_flying1, indoor_flying2 and indoor_flying3, respectively.