Redefining Radar Segmentation: Simultaneous Static-Moving Segmentation and Ego-Motion Estimation using Radar Point Clouds

TL;DR

This work redefines radar perception by jointly performing static/moving segmentation and ego-motion estimation directly from unprocessed radar point clouds, without requiring cloud aggregation or external odometry. It introduces a lightweight neural network with spatialMLP and temporal GRU backbones and a dual-head prediction scheme complemented by two weight-update heads, enabling instantaneous predictions over a moving window of frames ( $T=8$ ). Using RadarScenes and novel evaluation protocols, the approach achieves a moving-object IoU of about $0.86$ and F1 around $0.92$, while delivering a precise ego-motion estimate with $RTE_{50}$ near $1.8$ m, all with roughly $0.15$ million parameters. The results demonstrate the feasibility and practical impact of single-frame radar-based dual-task perception, offering immediate benefits for downstream tasks such as tracking, SLAM, and mapping in adverse weather and low-light conditions.

Abstract

Conventional radar segmentation research has typically focused on learning category labels for different moving objects. Although fundamental differences between radar and optical sensors lead to differences in the reliability of predicting accurate and consistent category labels, a review of common radar perception tasks in automotive reveals that determining whether an object is moving or static is a prerequisite for most tasks. To fill this gap, this study proposes a neural network based solution that can simultaneously segment static and moving objects from radar point clouds. Furthermore, since the measured radial velocity of static objects is correlated with the motion of the radar, this approach can also estimate the instantaneous 2D velocity of the moving platform or vehicle (ego motion). However, despite performing dual tasks, the proposed method employs very simple yet effective building blocks for feature extraction: multi layer perceptrons (MLPs) and recurrent neural networks (RNNs). In addition to being the first of its kind in the literature, the proposed method also demonstrates the feasibility of extracting the information required for the dual task directly from unprocessed point clouds, without the need for cloud aggregation, Doppler compensation, motion compensation, or any other intermediate signal processing steps. To measure its performance, this study introduces a set of novel evaluation metrics and tests the proposed method using a challenging real world radar dataset, RadarScenes. The results show that the proposed method not only performs well on the dual tasks, but also has broad application potential in other radar perception tasks.

Figures (9)

• Figure 1: The proposed method takes multidimensional radar point clouds as input, uses neural networks (NNs) for automatic feature extraction, and then segments static and moving objects. Based on the measured radial velocity of static objects, the method can estimate the ego-motion of the moving vehicle. The distinct moving instances can also be generated after applying a clustering algorithm to the predicted moving objects. In this example, the RadarScenes radar_scenes_dataset dataset is used for testing.
• Figure 2: Architecture of the proposed neural network for simultaneous static-moving object segmentation and vehicle ego-motion estimation. The network takes multidimensional radar point clouds as input, performs automatic spatial-temporal feature extraction, predicts static labels and moving labels for each detection point, and implements the weighted least squares (w-LSQ) for ego-motion estimation. As an illustrative application, moving instances can be generated after applying a clustering algorithm to the grouped moving objects.
• Figure 3: An illustration of how moving and static objects appear in radar point clouds across multiple consecutive frames. The first row shows the polar profile, which presents the radar point cloud in the Range-AoA domain. The moving objects, marked in red, exhibit clear spatial concentration and temporal correlation in the polar profile. The second row shows the Doppler profile, which presents the radar point cloud in the radial velocity-AoA domain. The static objects, marked in red, exhibit distinct sine-like spatial pattern with little temporal variation. In this example, the RadarScenes dataset radar_scenes_dataset is used.
• Figure 4: An illustration of how the initial weights for static and moving objects are updated in the two weight update heads. The yellow blocks represent the static update head, and the cyan blocks represent the moving update head. In this example, the RadarScenes dataset radar_scenes_dataset is used, and the plots show the radar point cloud in the radial velocity-AoA domain.
• Figure 5: The performance of the proposed method for moving object segmentation and ego-motion estimation with different lengths of the input moving window (in radar frames). The blue solid line represents the missed detection rate of the model, and the red solid line represents the RET_50.