VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow

TL;DR

VoteFlow tackles self-supervised scene flow estimation by embedding local rigidity as an architectural inductive bias through a differentiable Voting Module. Operating on pillar-based feature maps, it discretizes a translation space and aggregates votes from neighboring pillars to identify shared motion, producing a per-pillar voting feature for end-to-end learning. Empirical results on Argoverse 2 and Waymo Open show VoteFlow achieving state-of-the-art performance among self-supervised methods and strong cross-dataset generalization, with fast inference (~25 ms per sample on an A100). By enforcing local rigidity directly in the network, VoteFlow reduces reliance on post-processing or heavy regularizers, advancing robust scene flow estimation for autonomous driving.

Abstract

Scene flow estimation aims to recover per-point motion from two adjacent LiDAR scans. However, in real-world applications such as autonomous driving, points rarely move independently of others, especially for nearby points belonging to the same object, which often share the same motion. Incorporating this locally rigid motion constraint has been a key challenge in self-supervised scene flow estimation, which is often addressed by post-processing or appending extra regularization. While these approaches are able to improve the rigidity of predicted flows, they lack an architectural inductive bias for local rigidity within the model structure, leading to suboptimal learning efficiency and inferior performance. In contrast, we enforce local rigidity with a lightweight add-on module in neural network design, enabling end-to-end learning. We design a discretized voting space that accommodates all possible translations and then identify the one shared by nearby points by differentiable voting. Additionally, to ensure computational efficiency, we operate on pillars rather than points and learn representative features for voting per pillar. We plug the Voting Module into popular model designs and evaluate its benefit on Argoverse 2 and Waymo datasets. We outperform baseline works with only marginal compute overhead. Code is available at https://github.com/tudelft-iv/VoteFlow.

Figures (4)

• Figure 1: Voting for identifying shared motion. We take inspiration from motion rigidity and identify a motion shared by the majority via voting. We design a discrete voting space that encapsulates all possible translations that might occur within time $\Delta t$. For a given star $\bigstar$ at time $t$, we calculate the displacements between itself and its neighbors $\bullet$ at time $t + \Delta t$ and cast votes, defined by the cosine feature similarity between $\bigstar$ and $\bullet$, to corresponding bins in the voting space. We accumulate votes from multiple spatially nearby points $\bigstar$ at time $t$. The voting result indicates the likelihood of a motion shared by nearby points. Our differentiable voting is a light-weight add-on module compatible with popular model designs in scene flow estimation. In practice, voting takes pillars as input rather than individual points, thus reducing the computation substantially.
• Figure 2: The overall architecture of VoteFlow. VoteFlow introduces a new end-to-end optimizable Voting Module that matches features in the local region around each pillar. Such matches are used to vote for a translation of the pillar. A CNN module summarizes the resulting votes as a 'voting feature'. The voting features are the output of the module and passed on to the decoder.
• Figure 3: Qualitative results on Argoverse 2 validation set. Colors indicate directions and saturation of the color indicates the scale of the flow estimation. Thanks to the local rigidity prior, our VoteFlow predicts more consistent and coherent flow over objects compared to our baseline.
• Figure 4: Visualization of the voting space. We select a pillar representing a moving object and plot its voting space, where the center indicates zero translations and boundaries indicate minimal and maximal translations along both dimensions. As shown in the voting space, the red arrow aligns with the ground truth flow (up to quantization errors). Overall, the voting space depicts the heatmap of the object's motion. For example, in the top row, the object is expected to move towards the bottom right, and the predicted heat map also has high responses along the same direction.