Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Prediction

Kangan Qian, Jinyu Miao, Ziang Luo, Zheng Fu, and Jinchen Li, Yining Shi, Yunlong Wang, Kun Jiang, Mengmeng Yang, Diange Yang

TL;DR

LEGO-Motion tackles class-agnostic motion prediction by integrating instance-level semantics into occupancy BEV grids. It introduces two modules: Interaction-Augmented Instance Encoder (IaIE) to model inter-agent relations and Instance-enhanced BEV Encoder (IeBE) to inject instance priors into BEV representations. Through experiments on nuScenes and FMCW LiDAR benchmarks, the method achieves state-of-the-art motion prediction accuracy while preserving real-time efficiency, outperforming prior occupancy-based methods by meaningful margins. The work demonstrates that combining instance-aware reasoning with grid-based representations yields robust, physics-consistent motion understanding in open-world driving scenarios.

Abstract

Accurate and reliable spatial and motion information plays a pivotal role in autonomous driving systems. However, object-level perception models struggle with handling open scenario categories and lack precise intrinsic geometry. On the other hand, occupancy-based class-agnostic methods excel in representing scenes but fail to ensure physics consistency and ignore the importance of interactions between traffic participants, hindering the model's ability to learn accurate and reliable motion. In this paper, we introduce a novel occupancy-instance modeling framework for class-agnostic motion prediction tasks, named LEGO-Motion, which incorporates instance features into Bird's Eye View (BEV) space. Our model comprises (1) a BEV encoder, (2) an Interaction-Augmented Instance Encoder, and (3) an Instance-Enhanced BEV Encoder, improving both interaction relationships and physics consistency within the model, thereby ensuring a more accurate and robust understanding of the environment. Extensive experiments on the nuScenes dataset demonstrate that our method achieves state-of-the-art performance, outperforming existing approaches. Furthermore, the effectiveness of our framework is validated on the advanced FMCW LiDAR benchmark, showcasing its practical applicability and generalization capabilities. The code will be made publicly available to facilitate further research.

LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Prediction

TL;DR

LEGO-Motion tackles class-agnostic motion prediction by integrating instance-level semantics into occupancy BEV grids. It introduces two modules: Interaction-Augmented Instance Encoder (IaIE) to model inter-agent relations and Instance-enhanced BEV Encoder (IeBE) to inject instance priors into BEV representations. Through experiments on nuScenes and FMCW LiDAR benchmarks, the method achieves state-of-the-art motion prediction accuracy while preserving real-time efficiency, outperforming prior occupancy-based methods by meaningful margins. The work demonstrates that combining instance-aware reasoning with grid-based representations yields robust, physics-consistent motion understanding in open-world driving scenarios.

Abstract

Accurate and reliable spatial and motion information plays a pivotal role in autonomous driving systems. However, object-level perception models struggle with handling open scenario categories and lack precise intrinsic geometry. On the other hand, occupancy-based class-agnostic methods excel in representing scenes but fail to ensure physics consistency and ignore the importance of interactions between traffic participants, hindering the model's ability to learn accurate and reliable motion. In this paper, we introduce a novel occupancy-instance modeling framework for class-agnostic motion prediction tasks, named LEGO-Motion, which incorporates instance features into Bird's Eye View (BEV) space. Our model comprises (1) a BEV encoder, (2) an Interaction-Augmented Instance Encoder, and (3) an Instance-Enhanced BEV Encoder, improving both interaction relationships and physics consistency within the model, thereby ensuring a more accurate and robust understanding of the environment. Extensive experiments on the nuScenes dataset demonstrate that our method achieves state-of-the-art performance, outperforming existing approaches. Furthermore, the effectiveness of our framework is validated on the advanced FMCW LiDAR benchmark, showcasing its practical applicability and generalization capabilities. The code will be made publicly available to facilitate further research.

Paper Structure

This paper contains 15 sections, 13 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Motion Prediction in BEV grid cells
  4. Mask Prediction
  5. Methodology
  6. Problem Formulation
  7. BEV Encoder
  8. Interaction-augmented Instance Encoder
  9. Instance-enhanced BEV Encoder
  10. Task-specific Heads and Loss Function
  11. Experiment
  12. Experimental Setup
  13. Main Results
  14. Ablation Study
  15. Conclusion

Figures (4)

  • Figure 1: Comparison between LEGO-Motion and previous motion prediction framework. Top row: Previous framework, which adapts encoder-decoder pipeline to capture and predict motion fields. Bottom: Our proposed LEGO-Motion.
  • Figure 2: Architecture of learning-enhanced grids for with occupancy instance modeling(LEGO-Motion). Top: Instance-enhanced BEV Encoder. Bottom left: Interaction-augmented Instance Encoder. Bottom right: Task-specific Heads.
  • Figure 3: Comparison between grid-based methods with and without LEGO-Motion. The radius of the circle represents the number of model parameters. (a) LEGO-Motion outperforms the baseline in traditional mean speed error; (b) It also demonstrates new capabilities in motion stability, with only a slight increase in model parameters.
  • Figure 4: Qualitative results of the proposed LEGO-Motion framework. Each group represents a specific traffic scenario. (a) Object-level ground truth (GT) in Bird's-Eye View (BEV); (b) Grid-level GT; (c) Motion prediction results. Motions are represented by arrows attached to each grid, and cell classification results are indicated by various colors: cyan for background, pink for vehicles, black for pedestrians, yellow for bikes, and red for others.