Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simulating an Autonomous System in CARLA using ROS 2

Joseph Abdo, Aditya Shibu, Moaiz Saeed, Abdul Maajid Aga, Apsara Sivaprazad, Mohamed Al-Musleh

TL;DR

This paper presents a ROS 2–driven autonomous racing stack implemented in CARLA that fuses LiDAR, stereo vision, GNSS, and IMU to detect track cones, localize, plan, and control a vehicle at high speeds for FS-AI 2025. The approach combines YOLOv8-based cone detection, low-level LiDAR–camera fusion, Cartographer SLAM with GNSS–IMU EKF fusion, and a Pure Pursuit/PID control loop, delivering robust perception and planning in realistic settings. Key contributions include high-fidelity perception fusion with adaptive confidence weighting, a depth estimation scheme that leverages stereo and temporal information, and a tightly coupled SLAM pipeline suitable for real-time autonomous racing. The work demonstrates strong simulation-based validation, achieving 96% cone-detection accuracy and 71–83 Hz processing for perception, and establishes a transferable pipeline toward deployment on hardware such as Jetson AGX Orin and ADS-DV platforms, enabling rapid development and testing of autonomous racing stacks.

Abstract

Autonomous racing offers a rigorous setting to stress test perception, planning, and control under high speed and uncertainty. This paper proposes an approach to design and evaluate a software stack for an autonomous race car in CARLA: Car Learning to Act simulator, targeting competitive driving performance in the Formula Student UK Driverless (FS-AI) 2025 competition. By utilizing a 360° light detection and ranging (LiDAR), stereo camera, global navigation satellite system (GNSS), and inertial measurement unit (IMU) sensor via ROS 2 (Robot Operating System), the system reliably detects the cones marking the track boundaries at distances of up to 35 m. Optimized trajectories are computed considering vehicle dynamics and simulated environmental factors such as visibility and lighting to navigate the track efficiently. The complete autonomous stack is implemented in ROS 2 and validated extensively in CARLA on a dedicated vehicle (ADS-DV) before being ported to the actual hardware, which includes the Jetson AGX Orin 64GB, ZED2i Stereo Camera, Robosense Helios 16P LiDAR, and CHCNAV Inertial Navigation System (INS).

Simulating an Autonomous System in CARLA using ROS 2

TL;DR

This paper presents a ROS 2–driven autonomous racing stack implemented in CARLA that fuses LiDAR, stereo vision, GNSS, and IMU to detect track cones, localize, plan, and control a vehicle at high speeds for FS-AI 2025. The approach combines YOLOv8-based cone detection, low-level LiDAR–camera fusion, Cartographer SLAM with GNSS–IMU EKF fusion, and a Pure Pursuit/PID control loop, delivering robust perception and planning in realistic settings. Key contributions include high-fidelity perception fusion with adaptive confidence weighting, a depth estimation scheme that leverages stereo and temporal information, and a tightly coupled SLAM pipeline suitable for real-time autonomous racing. The work demonstrates strong simulation-based validation, achieving 96% cone-detection accuracy and 71–83 Hz processing for perception, and establishes a transferable pipeline toward deployment on hardware such as Jetson AGX Orin and ADS-DV platforms, enabling rapid development and testing of autonomous racing stacks.

Abstract

Autonomous racing offers a rigorous setting to stress test perception, planning, and control under high speed and uncertainty. This paper proposes an approach to design and evaluate a software stack for an autonomous race car in CARLA: Car Learning to Act simulator, targeting competitive driving performance in the Formula Student UK Driverless (FS-AI) 2025 competition. By utilizing a 360° light detection and ranging (LiDAR), stereo camera, global navigation satellite system (GNSS), and inertial measurement unit (IMU) sensor via ROS 2 (Robot Operating System), the system reliably detects the cones marking the track boundaries at distances of up to 35 m. Optimized trajectories are computed considering vehicle dynamics and simulated environmental factors such as visibility and lighting to navigate the track efficiently. The complete autonomous stack is implemented in ROS 2 and validated extensively in CARLA on a dedicated vehicle (ADS-DV) before being ported to the actual hardware, which includes the Jetson AGX Orin 64GB, ZED2i Stereo Camera, Robosense Helios 16P LiDAR, and CHCNAV Inertial Navigation System (INS).

Paper Structure

This paper contains 21 sections, 3 equations, 10 figures.

Table of Contents

  1. Literature Review
  2. Methodology
  3. System Architecture
  4. Perception
  5. Transform frames
  6. LiDAR based Cone Perception
  7. Camera based Cone Perception
  8. Model Development and Training
  9. Performance metrics and Evaluation
  10. Depth Camera (vision-based)
  11. Advanced Depth Estimation
  12. LiDAR-Camera sensor Fusion
  13. Low-level Fusion Approach
  14. Sensor Alignment and Calibration
  15. Data Association and Fusion Algorithm
  16. ...and 6 more sections

Figures (10)

  • Figure 1: Integrated Sensor-to-Control Architecture for Autonomous Navigation.
  • Figure 2: TF2 frame tree of the autonomous system
  • Figure 3: Real-world ADS-DV platform implementation utilizing the custom YOLOv8 model
  • Figure 4: Confusion Matrix of the trained YOLOv8 Model
  • Figure 5: Precision-recall curve
  • ...and 5 more figures