Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiCAR: pseudo-RGB LiDAR image for CAR segmentation

Ignacio de Loyola Páez-Ubieta, Edison P. Velasco-Sánchez, Santiago T. Puente

TL;DR

Segmenting cars in pseudo-RGB images derived from LiDAR data is addressed by generating a pseudo-RGB representation through fusion of LiDAR channels and evaluating fast single-stage instance segmentation networks. The approach trains YOLOv5/7/8 on the LiDAR-derived imagery and demonstrates that YOLOv8 large yields strong detection (≈0.88 precision, ≈0.81 recall, ≈0.881 mAP@0.5) and mask segmentation (≈0.826 mAP, ≈0.815 precision), while enabling real-time on-board inference (≈33–66 ms). Real-world tests incorporate BoT-SORT and ByteTrack trackers to maintain instance identities across frames, with BoT-SORT preferred for enhanced tracking robustness. The work provides a public dataset and shows that LiDAR-derived pseudo-RGB inputs can support accurate, real-time car segmentation and tracking in outdoor settings, paving the way for broader object categories and calibration-based refinements in future work.

Abstract

With the advancement of computing resources, an increasing number of Neural Networks (NNs) are appearing for image detection and segmentation appear. However, these methods usually accept as input a RGB 2D image. On the other side, Light Detection And Ranging (LiDAR) sensors with many layers provide images that are similar to those obtained from a traditional low resolution RGB camera. Following this principle, a new dataset for segmenting cars in pseudo-RGB images has been generated. This dataset combines the information given by the LiDAR sensor into a Spherical Range Image (SRI), concretely the reflectivity, near infrared and signal intensity 2D images. These images are then fed into instance segmentation NNs. These NNs segment the cars that appear in these images, having as result a Bounding Box (BB) and mask precision of 88% and 81.5% respectively with You Only Look Once (YOLO)-v8 large. By using this segmentation NN, some trackers have been applied so as to follow each car segmented instance along a video feed, having great performance in real world experiments.

LiCAR: pseudo-RGB LiDAR image for CAR segmentation

TL;DR

Segmenting cars in pseudo-RGB images derived from LiDAR data is addressed by generating a pseudo-RGB representation through fusion of LiDAR channels and evaluating fast single-stage instance segmentation networks. The approach trains YOLOv5/7/8 on the LiDAR-derived imagery and demonstrates that YOLOv8 large yields strong detection (≈0.88 precision, ≈0.81 recall, ≈0.881 mAP@0.5) and mask segmentation (≈0.826 mAP, ≈0.815 precision), while enabling real-time on-board inference (≈33–66 ms). Real-world tests incorporate BoT-SORT and ByteTrack trackers to maintain instance identities across frames, with BoT-SORT preferred for enhanced tracking robustness. The work provides a public dataset and shows that LiDAR-derived pseudo-RGB inputs can support accurate, real-time car segmentation and tracking in outdoor settings, paving the way for broader object categories and calibration-based refinements in future work.

Abstract

With the advancement of computing resources, an increasing number of Neural Networks (NNs) are appearing for image detection and segmentation appear. However, these methods usually accept as input a RGB 2D image. On the other side, Light Detection And Ranging (LiDAR) sensors with many layers provide images that are similar to those obtained from a traditional low resolution RGB camera. Following this principle, a new dataset for segmenting cars in pseudo-RGB images has been generated. This dataset combines the information given by the LiDAR sensor into a Spherical Range Image (SRI), concretely the reflectivity, near infrared and signal intensity 2D images. These images are then fed into instance segmentation NNs. These NNs segment the cars that appear in these images, having as result a Bounding Box (BB) and mask precision of 88% and 81.5% respectively with You Only Look Once (YOLO)-v8 large. By using this segmentation NN, some trackers have been applied so as to follow each car segmented instance along a video feed, having great performance in real world experiments.
Paper Structure (14 sections, 4 figures, 4 tables)

This paper contains 14 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Image generation
  4. Segmentation NNs
  5. Experiments
  6. Setup
  7. Dataset
  8. NN training
  9. Results
  10. Test set
  11. Real experiments
  12. Conclusion
  13. Acknowledgments.
  14. Disclosure of Interests.

Figures (4)

  • Figure 1: Combination of LiDAR channels for generating the pseudo-RGB image.
  • Figure 2: robot with both the Ouster OS1-128 3D- and the on board computer.
  • Figure 3: Some samples from test set segmented by -v8 large.
  • Figure 4: Several frames running -v8 large using BoT-SORT as tracker.