Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task-Oriented Pre-Training for Drivable Area Detection

Fulong Ma, Guoyang Zhao, Weiqing Qi, Ming Liu, Jun Ma

TL;DR

This paper targets drivable area detection by introducing task-oriented pre-training that generates coarse in-domain labels with a frozen SAM segmentation model and then refines selection using a SCEF-tuned CLIP to identify drivable-area proposals. The pre-training data are created from SAM outputs and subsequently fine-tuned with manual KITTI road annotations, enabling the model to learn task-specific features with high efficiency. Experiments on the KITTI road dataset show consistent improvements over ImageNet-based pre-training and competitive performance against self-training methods, with substantial gains in mIoU and F1-score and significantly reduced training time and resource demands. The approach offers practical benefits for autonomous driving systems by delivering strong drivable area detection performance at low computational cost and data requirements.

Abstract

Pre-training techniques play a crucial role in deep learning, enhancing models' performance across a variety of tasks. By initially training on large datasets and subsequently fine-tuning on task-specific data, pre-training provides a solid foundation for models, improving generalization abilities and accelerating convergence rates. This approach has seen significant success in the fields of natural language processing and computer vision. However, traditional pre-training methods necessitate large datasets and substantial computational resources, and they can only learn shared features through prolonged training and struggle to capture deeper, task-specific features. In this paper, we propose a task-oriented pre-training method that begins with generating redundant segmentation proposals using the Segment Anything (SAM) model. We then introduce a Specific Category Enhancement Fine-tuning (SCEF) strategy for fine-tuning the Contrastive Language-Image Pre-training (CLIP) model to select proposals most closely related to the drivable area from those generated by SAM. This approach can generate a lot of coarse training data for pre-training models, which are further fine-tuned using manually annotated data, thereby improving model's performance. Comprehensive experiments conducted on the KITTI road dataset demonstrate that our task-oriented pre-training method achieves an all-around performance improvement compared to models without pre-training. Moreover, our pre-training method not only surpasses traditional pre-training approach but also achieves the best performance compared to state-of-the-art self-training methods.

Task-Oriented Pre-Training for Drivable Area Detection

TL;DR

This paper targets drivable area detection by introducing task-oriented pre-training that generates coarse in-domain labels with a frozen SAM segmentation model and then refines selection using a SCEF-tuned CLIP to identify drivable-area proposals. The pre-training data are created from SAM outputs and subsequently fine-tuned with manual KITTI road annotations, enabling the model to learn task-specific features with high efficiency. Experiments on the KITTI road dataset show consistent improvements over ImageNet-based pre-training and competitive performance against self-training methods, with substantial gains in mIoU and F1-score and significantly reduced training time and resource demands. The approach offers practical benefits for autonomous driving systems by delivering strong drivable area detection performance at low computational cost and data requirements.

Abstract

Pre-training techniques play a crucial role in deep learning, enhancing models' performance across a variety of tasks. By initially training on large datasets and subsequently fine-tuning on task-specific data, pre-training provides a solid foundation for models, improving generalization abilities and accelerating convergence rates. This approach has seen significant success in the fields of natural language processing and computer vision. However, traditional pre-training methods necessitate large datasets and substantial computational resources, and they can only learn shared features through prolonged training and struggle to capture deeper, task-specific features. In this paper, we propose a task-oriented pre-training method that begins with generating redundant segmentation proposals using the Segment Anything (SAM) model. We then introduce a Specific Category Enhancement Fine-tuning (SCEF) strategy for fine-tuning the Contrastive Language-Image Pre-training (CLIP) model to select proposals most closely related to the drivable area from those generated by SAM. This approach can generate a lot of coarse training data for pre-training models, which are further fine-tuned using manually annotated data, thereby improving model's performance. Comprehensive experiments conducted on the KITTI road dataset demonstrate that our task-oriented pre-training method achieves an all-around performance improvement compared to models without pre-training. Moreover, our pre-training method not only surpasses traditional pre-training approach but also achieves the best performance compared to state-of-the-art self-training methods.
Paper Structure (17 sections, 4 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. Drivable Area Detection
  4. Foundation Models
  5. Pre-Training and Self-Training
  6. METHOD
  7. Redundant Masks Generation Using SAM
  8. Specific Category Enhancement Fine-Tuning
  9. EXPERIMENT
  10. Experiment Details
  11. Experiment Setup
  12. Dataset
  13. Evaluation Metrics
  14. Performance Evaluation
  15. Comparison with Pre-Training
  16. ...and 2 more sections

Figures (4)

  • Figure 1: A visual qualitative comparison of the results using our task-oriented pre-training method versus without pre-training across several metrics: mIoU, Accuracy, Precision, Recall, and F1-Score.
  • Figure 2: The overall architecture of our method.
  • Figure 3: Segmentation results of the SAM model in the "everything” mode, with the image patch within the red box representing pixels that belong to the drivable area.
  • Figure 4: The qualitative visual comparison results between the training labels generated by our method and the KITTI road ground truth. Figure (a) shows some examples that are very close to the ground truth, while Figure (b) shows some failure cases.The true positive, false negative and false positive pixels are shown in green, red and blue, respectively.