Event-based Structure-from-Orbit

TL;DR

The aim is to simultaneously reconstruct the 3D structure of a fast spinning object observed from a static event camera, and recover the equivalent orbital motion of the camera, in an event-based structure-from-orbit (eSfO).

Abstract

Event sensors offer high temporal resolution visual sensing, which makes them ideal for perceiving fast visual phenomena without suffering from motion blur. Certain applications in robotics and vision-based navigation require 3D perception of an object undergoing circular or spinning motion in front of a static camera, such as recovering the angular velocity and shape of the object. The setting is equivalent to observing a static object with an orbiting camera. In this paper, we propose event-based structure-from-orbit (eSfO), where the aim is to simultaneously reconstruct the 3D structure of a fast spinning object observed from a static event camera, and recover the equivalent orbital motion of the camera. Our contributions are threefold: since state-of-the-art event feature trackers cannot handle periodic self-occlusion due to the spinning motion, we develop a novel event feature tracker based on spatio-temporal clustering and data association that can better track the helical trajectories of valid features in the event data. The feature tracks are then fed to our novel factor graph-based structure-from-orbit back-end that calculates the orbital motion parameters (e.g., spin rate, relative rotational axis) that minimize the reprojection error. For evaluation, we produce a new event dataset of objects under spinning motion. Comparisons against ground truth indicate the efficacy of eSfO.

Figures (11)

• Figure 1: In eSfO, we exploit the equivalence between a static event camera observing a spinning object, and an orbiting event camera observing a static object. This enables us to jointly estimate the motion parameters (e.g., spin rate, rotational axis relative to the camera), as well as the sparse structure of the object.
• Figure 2: L: Problem setup. R: Visualization of eSfO parameters in the orbit view of the problem. $w$ is an arbitrary world frame.
• Figure 3: The proposed eSfO pipeline.
• Figure 4: The formulation of the orbit factor (OF) illustrated for a single 3D point $\mathbf{x}_i$ and its corresponding feature positions for three different timestamps. Global parameters are highlighted as a group at the bottom of the figure. Estimated quantities are in green and inputs are in white squares.
• Figure 5: Example event frames from our dataset. soho-sideon-fast (top left), hubble-diagonal-med (top right), switch-perpendicular-slow (bottom left) and dualshock-topdown-med (bottom right).