Flow4D: Leveraging 4D Voxel Network for LiDAR Scene Flow Estimation

TL;DR

Flow4D tackles the problem of accurately estimating LiDAR scene flow by explicitly modeling spatio-temporal features through a 4D voxel network. It temporally fuses multiple point clouds after a VFE and uses Spatio-Temporal Decomposition Blocks (STDB) to replace heavy 4D convolutions with efficient 3D spatial plus 1D temporal operations, enabling real-time performance. Evaluated on Argoverse 2, Flow4D achieves state-of-the-art results, notably a 45.9% improvement in mean Dynamic Normalized Endpoint Error over prior methods, and benefits further from five-frame temporal context. The work demonstrates the practical impact of explicit 4D feature extraction for robust motion understanding in autonomous driving and points toward future hardware-efficient deployments.

Abstract

Understanding the motion states of the surrounding environment is critical for safe autonomous driving. These motion states can be accurately derived from scene flow, which captures the three-dimensional motion field of points. Existing LiDAR scene flow methods extract spatial features from each point cloud and then fuse them channel-wise, resulting in the implicit extraction of spatio-temporal features. Furthermore, they utilize 2D Bird's Eye View and process only two frames, missing crucial spatial information along the Z-axis and the broader temporal context, leading to suboptimal performance. To address these limitations, we propose Flow4D, which temporally fuses multiple point clouds after the 3D intra-voxel feature encoder, enabling more explicit extraction of spatio-temporal features through a 4D voxel network. However, while using 4D convolution improves performance, it significantly increases the computational load. For further efficiency, we introduce the Spatio-Temporal Decomposition Block (STDB), which combines 3D and 1D convolutions instead of using heavy 4D convolution. In addition, Flow4D further improves performance by using five frames to take advantage of richer temporal information. As a result, the proposed method achieves a 45.9% higher performance compared to the state-of-the-art while running in real-time, and won 1st place in the 2024 Argoverse 2 Scene Flow Challenge. The code is available at https://github.com/dgist-cvlab/Flow4D.

Figures (4)

• Figure 1: Fusion strategy comparison for LiDAR scene flow estimation. (a) Existing methods jund2022fastflow3dzhang2024deflow use a 2D pillar representation and find correspondences via channel fusion followed by a 2D decoder. However, these methods only allow limited channel-level interaction due to their 2D representation. (b) In contrast, our proposed method allows explicit spatio-temporal interaction to find accurate correspondences by using a 4D voxel representation.
• Figure 2: Overall framework of the Flow4D. Sequential point clouds are processed through individual Voxel Feature Encoders to extract intra-voxel features. These features are then temporally fused to form 4D voxel features. The 4D voxel network, which consists of Spatio-Temporal Decomposition Blocks (STDBs), extracts voxel-wise spatio-temporal features. Finally, the Point Head module estimates point-wise flow vectors.
• Figure 3: Comparison of spatio-temporal feature extraction blocks.
• Figure 4: Qualitative comparison on Argoverse 2 validation set. Following DeFlow zhang2024deflow, we represent the direction and magnitude of each motion vector as hue and saturation, respectively. Our proposed method estimates motion vectors more accurately than existing methods jund2022fastflow3dzhang2024deflow for all classes, including pedestrians, vehicles, trucks, and buses.