Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain Generalization in Autonomous Driving: Evaluating YOLOv8s, RT-DETR, and YOLO-NAS with the ROAD-Almaty Dataset

Madiyar Alimov, Temirlan Meiramkhanov

TL;DR

This paper examines how well three state-of-the-art detectors generalize to autonomous driving scenarios in Kazakhstan without retraining, using the ROAD-Almaty dataset. It systematically compares YOLOv8s, RT-DETR, and YOLO-NAS under diverse weather and lighting, reporting that RT-DETR achieves the strongest robustness at $IoU=0.5$ with an average F1 of approximately $0.672$, while YOLOv8s and YOLO-NAS lag behind. All models exhibit substantial performance drops when localization becomes stricter ($IoU=0.75$) and under challenging conditions like heavy snowfall or low light, underscoring a persistent domain-sh shift problem. The study highlights the need for geographically diverse training data and domain adaptation techniques to enable reliable, global autonomous driving systems.

Abstract

This study investigates the domain generalization capabilities of three state-of-the-art object detection models - YOLOv8s, RT-DETR, and YOLO-NAS - within the unique driving environment of Kazakhstan. Utilizing the newly constructed ROAD-Almaty dataset, which encompasses diverse weather, lighting, and traffic conditions, we evaluated the models' performance without any retraining. Quantitative analysis revealed that RT-DETR achieved an average F1-score of 0.672 at IoU=0.5, outperforming YOLOv8s (0.458) and YOLO-NAS (0.526) by approximately 46% and 27%, respectively. Additionally, all models exhibited significant performance declines at higher IoU thresholds (e.g., a drop of approximately 20% when increasing IoU from 0.5 to 0.75) and under challenging environmental conditions, such as heavy snowfall and low-light scenarios. These findings underscore the necessity for geographically diverse training datasets and the implementation of specialized domain adaptation techniques to enhance the reliability of autonomous vehicle detection systems globally. This research contributes to the understanding of domain generalization challenges in autonomous driving, particularly in underrepresented regions.

Domain Generalization in Autonomous Driving: Evaluating YOLOv8s, RT-DETR, and YOLO-NAS with the ROAD-Almaty Dataset

TL;DR

This paper examines how well three state-of-the-art detectors generalize to autonomous driving scenarios in Kazakhstan without retraining, using the ROAD-Almaty dataset. It systematically compares YOLOv8s, RT-DETR, and YOLO-NAS under diverse weather and lighting, reporting that RT-DETR achieves the strongest robustness at with an average F1 of approximately , while YOLOv8s and YOLO-NAS lag behind. All models exhibit substantial performance drops when localization becomes stricter () and under challenging conditions like heavy snowfall or low light, underscoring a persistent domain-sh shift problem. The study highlights the need for geographically diverse training data and domain adaptation techniques to enable reliable, global autonomous driving systems.

Abstract

This study investigates the domain generalization capabilities of three state-of-the-art object detection models - YOLOv8s, RT-DETR, and YOLO-NAS - within the unique driving environment of Kazakhstan. Utilizing the newly constructed ROAD-Almaty dataset, which encompasses diverse weather, lighting, and traffic conditions, we evaluated the models' performance without any retraining. Quantitative analysis revealed that RT-DETR achieved an average F1-score of 0.672 at IoU=0.5, outperforming YOLOv8s (0.458) and YOLO-NAS (0.526) by approximately 46% and 27%, respectively. Additionally, all models exhibited significant performance declines at higher IoU thresholds (e.g., a drop of approximately 20% when increasing IoU from 0.5 to 0.75) and under challenging environmental conditions, such as heavy snowfall and low-light scenarios. These findings underscore the necessity for geographically diverse training datasets and the implementation of specialized domain adaptation techniques to enhance the reliability of autonomous vehicle detection systems globally. This research contributes to the understanding of domain generalization challenges in autonomous driving, particularly in underrepresented regions.

Paper Structure

This paper contains 27 sections, 2 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Advances in Object Detection for Autonomous Driving
  4. The Challenge of Domain Shift
  5. Existing Datasets and Their Geographic Limitations
  6. Approaches to Domain Adaptation and Generalization
  7. State-of-the-Art Models and Unanswered Questions
  8. Identifying the Research Gap
  9. METHODOLOGY
  10. Data Collection and Preparation
  11. Annotation Protocol and Ethical Considerations
  12. Data Splits and Experimental Setup
  13. Evaluation Metrics and Tools
  14. Sample Images from ROAD-Almaty Dataset
  15. RESULTS
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Collage of sample frames from the ROAD-Almaty dataset under varying weather and lighting conditions.
  • Figure 2: Comparative F1-scores at IoU=0.5 across five test sequences (Seq1–Seq5) for YOLOv8s, RT-DETR, and YOLO-NAS. Each group of three bars represents a single sequence, illustrating how performance varies with changing environmental conditions and how RT-DETR consistently outperforms the YOLO-based models.