Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast and Efficient Transformer-based Method for Bird's Eye View Instance Prediction

Miguel Antunes-García, Luis M. Bergasa, Santiago Montiel-Marín, Rafael Barea, Fabio Sánchez-García, Ángel Llamazares

TL;DR

This paper introduces a novel BEV instance prediction architecture based on a simplified paradigm that relies only on instance segmentation and flow prediction, aiming at reduced parameter counts and inference times compared to existing SOTA architectures, thanks to the incorporation of an efficient transformer-based architecture.

Abstract

Accurate object detection and prediction are critical to ensure the safety and efficiency of self-driving architectures. Predicting object trajectories and occupancy enables autonomous vehicles to anticipate movements and make decisions with future information, increasing their adaptability and reducing the risk of accidents. Current State-Of-The-Art (SOTA) approaches often isolate the detection, tracking, and prediction stages, which can lead to significant prediction errors due to accumulated inaccuracies between stages. Recent advances have improved the feature representation of multi-camera perception systems through Bird's-Eye View (BEV) transformations, boosting the development of end-to-end systems capable of predicting environmental elements directly from vehicle sensor data. These systems, however, often suffer from high processing times and number of parameters, creating challenges for real-world deployment. To address these issues, this paper introduces a novel BEV instance prediction architecture based on a simplified paradigm that relies only on instance segmentation and flow prediction. The proposed system prioritizes speed, aiming at reduced parameter counts and inference times compared to existing SOTA architectures, thanks to the incorporation of an efficient transformer-based architecture. Furthermore, the implementation of the proposed architecture is optimized for performance improvements in PyTorch version 2.1. Code and trained models are available at https://github.com/miguelag99/Efficient-Instance-Prediction

Fast and Efficient Transformer-based Method for Bird's Eye View Instance Prediction

TL;DR

This paper introduces a novel BEV instance prediction architecture based on a simplified paradigm that relies only on instance segmentation and flow prediction, aiming at reduced parameter counts and inference times compared to existing SOTA architectures, thanks to the incorporation of an efficient transformer-based architecture.

Abstract

Accurate object detection and prediction are critical to ensure the safety and efficiency of self-driving architectures. Predicting object trajectories and occupancy enables autonomous vehicles to anticipate movements and make decisions with future information, increasing their adaptability and reducing the risk of accidents. Current State-Of-The-Art (SOTA) approaches often isolate the detection, tracking, and prediction stages, which can lead to significant prediction errors due to accumulated inaccuracies between stages. Recent advances have improved the feature representation of multi-camera perception systems through Bird's-Eye View (BEV) transformations, boosting the development of end-to-end systems capable of predicting environmental elements directly from vehicle sensor data. These systems, however, often suffer from high processing times and number of parameters, creating challenges for real-world deployment. To address these issues, this paper introduces a novel BEV instance prediction architecture based on a simplified paradigm that relies only on instance segmentation and flow prediction. The proposed system prioritizes speed, aiming at reduced parameter counts and inference times compared to existing SOTA architectures, thanks to the incorporation of an efficient transformer-based architecture. Furthermore, the implementation of the proposed architecture is optimized for performance improvements in PyTorch version 2.1. Code and trained models are available at https://github.com/miguelag99/Efficient-Instance-Prediction

Paper Structure

This paper contains 14 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Camera 3D object detection
  4. Motion and Instance Prediction
  5. Transformer-based architectures for dense tasks
  6. ARCHITECTURE
  7. Feature extraction and BEV projection
  8. Segmentation and flow branches
  9. Flow warping
  10. EXPERIMENTS
  11. Implementation and Datasets
  12. Losses and metrics
  13. Procedure and results
  14. CONCLUSIONS AND FUTURE WORKS

Figures (4)

  • Figure 1: Our proposed architecture uses a multi-camera system to predict the position of the instances in the scene. We achieve similar results as our baseline PowerBEV reducing the number of parameters and latencies.
  • Figure 2: Diagram of the proposed architecture. First, the features of all the images are extracted for the whole input sequence. Each set of features of each instant is projected to the BEV using the information generated in the depth channels. For the past frames, it is necessary to apply a transformation that translates them to a unified system in the present frame. The generated BEV feature map is applied to two branches that generate the backward flow and segmentation values.
  • Figure 3: Head architecture for the segmentation and flow branches.
  • Figure 4: Qualitative results on NuScenes validation set. Each detected instance is represented by a color. If future positions are predicted, they are represented with the same color but with transparency. The ego vehicle is represented in black at the center.