Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Event-based Mosaicing Bundle Adjustment

Shuang Guo, Guillermo Gallego

TL;DR

Event cameras enable asynchronous brightness changes, and EMBA addresses event-only mosaicing bundle adjustment for pure rotational motion by formulating a regularized non-linear least squares problem using the linearized event generation model (LEGM) and a panoramic gradient map. The approach exploits block-diagonal sparsity induced by LEGM to enable efficient Levenberg–Marquardt optimization via a Schur-complement-based solver, refining both camera orientations and the intensity panorama without converting events to image-like representations. Across synthetic and real datasets, EMBA achieves notable reductions in photometric error and improved map quality, including high-resolution panoramas at VGA/HD scales and robust performance when initialized from various front-ends. The work advances event-based SLAM by providing a scalable back-end that jointly optimizes pose and dense scene texture, with code made publicly available for broader use and benchmarking.

Abstract

We tackle the problem of mosaicing bundle adjustment (i.e., simultaneous refinement of camera orientations and scene map) for a purely rotating event camera. We formulate the problem as a regularized non-linear least squares optimization. The objective function is defined using the linearized event generation model in the camera orientations and the panoramic gradient map of the scene. We show that this BA optimization has an exploitable block-diagonal sparsity structure, so that the problem can be solved efficiently. To the best of our knowledge, this is the first work to leverage such sparsity to speed up the optimization in the context of event-based cameras, without the need to convert events into image-like representations. We evaluate our method, called EMBA, on both synthetic and real-world datasets to show its effectiveness (50% photometric error decrease), yielding results of unprecedented quality. In addition, we demonstrate EMBA using high spatial resolution event cameras, yielding delicate panoramas in the wild, even without an initial map. Project page: https://github.com/tub-rip/emba

Event-based Mosaicing Bundle Adjustment

TL;DR

Event cameras enable asynchronous brightness changes, and EMBA addresses event-only mosaicing bundle adjustment for pure rotational motion by formulating a regularized non-linear least squares problem using the linearized event generation model (LEGM) and a panoramic gradient map. The approach exploits block-diagonal sparsity induced by LEGM to enable efficient Levenberg–Marquardt optimization via a Schur-complement-based solver, refining both camera orientations and the intensity panorama without converting events to image-like representations. Across synthetic and real datasets, EMBA achieves notable reductions in photometric error and improved map quality, including high-resolution panoramas at VGA/HD scales and robust performance when initialized from various front-ends. The work advances event-based SLAM by providing a scalable back-end that jointly optimizes pose and dense scene texture, with code made publicly available for broader use and benchmarking.

Abstract

We tackle the problem of mosaicing bundle adjustment (i.e., simultaneous refinement of camera orientations and scene map) for a purely rotating event camera. We formulate the problem as a regularized non-linear least squares optimization. The objective function is defined using the linearized event generation model in the camera orientations and the panoramic gradient map of the scene. We show that this BA optimization has an exploitable block-diagonal sparsity structure, so that the problem can be solved efficiently. To the best of our knowledge, this is the first work to leverage such sparsity to speed up the optimization in the context of event-based cameras, without the need to convert events into image-like representations. We evaluate our method, called EMBA, on both synthetic and real-world datasets to show its effectiveness (50% photometric error decrease), yielding results of unprecedented quality. In addition, we demonstrate EMBA using high spatial resolution event cameras, yielding delicate panoramas in the wild, even without an initial map. Project page: https://github.com/tub-rip/emba
Paper Structure (29 sections, 23 equations, 11 figures, 10 tables)

This paper contains 29 sections, 23 equations, 11 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Event-based Mosaicing Bundle Adjustment
  4. Event Generation Model (EGM)
  5. Problem Formulation
  6. Experiments
  7. Experimental Setup
  8. Experiments on Synthetic Data
  9. Experiments on Real Data
  10. Relationship with CMax-SLAM
  11. Experiments with VGA and HD event cameras
  12. Complexity Analysis and Runtime
  13. Limitations
  14. Conclusion
  15. Supplementary Material
  16. ...and 14 more sections

Figures (11)

  • Figure 1: Our back-end module EMBA jointly refines the camera rotations and panoramic gradient map. The intensity map can be recovered by solving Poisson's equation.
  • Figure 2: Initial intensity map (top), and final map $M$ (middle), via the refined gradient map $\nabla M$ (bottom), for the street data from Guo24tro. Three insets are also shown.
  • Figure 3: Sparsity illustration. (a) Mask of valid pixels; (b) $1000 \times 1000$ block at the top left of matrix $\mathtt{A}$; (c) Zoomed-in version of $\mathtt{A}_{22}$ showing its block-diagonal structure.
  • Figure 4: Camera trajectory degrees-of-freedom (DOFs) before ("CMax-$\boldsymbol{\omega}$") and after ("CMax-$\boldsymbol{\omega}$+EMBA") refinement, for some synthetic and real sequences from Mueggler17ijrrGuo24tro.
  • Figure 5: EMBA results on synthetic data. Panoramic maps have $2048 \times 1024$ px. Initial camera rotations are obtained using CMax-$\boldsymbol{\omega}$Gallego17ral.
  • ...and 6 more figures