Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Infrastructure Sensor-enabled Vehicle Data Generation using Multi-Sensor Fusion for Proactive Safety Applications at Work Zone

Suhala Rabab Saba, Sakib Khan, Minhaj Uddin Ahmad, Jiahe Cao, Mizanur Rahman, Li Zhao, Nathan Huynh, Eren Erman Ozguven

TL;DR

This work addresses the challenge of deploying reliable, infrastructure-based vehicle sensing in work zones by integrating roadside cameras and LiDAR within a SUMO-CARLA co-simulation and applying a Kalman Filter–based late fusion to produce accurate vehicle trajectories. The camera pipeline uses YOLOv8 for detection and bounding-box tracking, with image-to-angular and pinhole-model mappings to world coordinates, while LiDAR processing relies on point-cloud clustering and a discrete KF for per-sensor smoothing. The core contribution is a decision-level fusion framework that dynamically weights camera and LiDAR measurements to yield robust trajectories, validated in both simulation and a field test with RTK-GPS ground truth; results show substantial longitudinal error reduction (up to $70\%$) and lateral accuracy of $1$–$3$ m, with resilience to intermittent sensor data. Practically, this work demonstrates a scalable, cost-effective pathway for infrastructure-enabled multi-sensor sensing to enable proactive safety measures in complex traffic environments, especially in active work zones where sensor data may be degraded or missing.

Abstract

Infrastructure-based sensing and real-time trajectory generation show promise for improving safety in high-risk roadway segments such as work zones, yet practical deployments are hindered by perspective distortion, complex geometry, occlusions, and costs. This study tackles these barriers by integrating roadside camera and LiDAR sensors into a cosimulation environment to develop a scalable, cost-effective vehicle detection and localization framework, and employing a Kalman Filter-based late fusion strategy to enhance trajectory consistency and accuracy. In simulation, the fusion algorithm reduced longitudinal error by up to 70 percent compared to individual sensors while preserving lateral accuracy within 1 to 3 meters. Field validation in an active work zone, using LiDAR, a radar-camera rig, and RTK-GPS as ground truth, demonstrated that the fused trajectories closely match real vehicle paths, even when single-sensor data are intermittent or degraded. These results confirm that KF based sensor fusion can reliably compensate for individual sensor limitations, providing precise and robust vehicle tracking capabilities. Our approach thus offers a practical pathway to deploy infrastructure-enabled multi-sensor systems for proactive safety measures in complex traffic environments.

Infrastructure Sensor-enabled Vehicle Data Generation using Multi-Sensor Fusion for Proactive Safety Applications at Work Zone

TL;DR

This work addresses the challenge of deploying reliable, infrastructure-based vehicle sensing in work zones by integrating roadside cameras and LiDAR within a SUMO-CARLA co-simulation and applying a Kalman Filter–based late fusion to produce accurate vehicle trajectories. The camera pipeline uses YOLOv8 for detection and bounding-box tracking, with image-to-angular and pinhole-model mappings to world coordinates, while LiDAR processing relies on point-cloud clustering and a discrete KF for per-sensor smoothing. The core contribution is a decision-level fusion framework that dynamically weights camera and LiDAR measurements to yield robust trajectories, validated in both simulation and a field test with RTK-GPS ground truth; results show substantial longitudinal error reduction (up to ) and lateral accuracy of – m, with resilience to intermittent sensor data. Practically, this work demonstrates a scalable, cost-effective pathway for infrastructure-enabled multi-sensor sensing to enable proactive safety measures in complex traffic environments, especially in active work zones where sensor data may be degraded or missing.

Abstract

Infrastructure-based sensing and real-time trajectory generation show promise for improving safety in high-risk roadway segments such as work zones, yet practical deployments are hindered by perspective distortion, complex geometry, occlusions, and costs. This study tackles these barriers by integrating roadside camera and LiDAR sensors into a cosimulation environment to develop a scalable, cost-effective vehicle detection and localization framework, and employing a Kalman Filter-based late fusion strategy to enhance trajectory consistency and accuracy. In simulation, the fusion algorithm reduced longitudinal error by up to 70 percent compared to individual sensors while preserving lateral accuracy within 1 to 3 meters. Field validation in an active work zone, using LiDAR, a radar-camera rig, and RTK-GPS as ground truth, demonstrated that the fused trajectories closely match real vehicle paths, even when single-sensor data are intermittent or degraded. These results confirm that KF based sensor fusion can reliably compensate for individual sensor limitations, providing precise and robust vehicle tracking capabilities. Our approach thus offers a practical pathway to deploy infrastructure-enabled multi-sensor systems for proactive safety measures in complex traffic environments.

Paper Structure

This paper contains 29 sections, 12 equations, 10 figures, 2 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. LITERATURE REVIEW
  4. METHOD
  5. Object Detection and Localization in Co-Simulation Environment
  6. Using Camera
  7. Step 1: Vehicle Detection and Tracking in CARLA
  8. The RGB camera sensor in CARLA captures video frames at a defined resolution and frame rate. Each frame is processed using the YOLOv8 deep learning model to detect vehicles. YOLO generates bounding boxes, class labels (e.g., "car", "truck"), and confidence scores. Object tracking is then applied to associate detections across consecutive frames, which enables the construction of vehicle trajectories.
  9. Step 2: Conversion from Image to Angular Coordinates
  10. The pixel coordinates P$(x_i, y_i)$ of each detected object are transformed into angular coordinates P$(x_\phi, y_\phi)$ through a relationship that accounts for the camera's field of view and image resolution. Specifically, each pixel position is mapped to an angle relative to the camera’s optical axis based on the horizontal and vertical angles subtended by the field of view. Using $x_i$ to $x_\phi$ as an example, the conversion can be done using Equation \ref{['eq1']}. In CARLA, the camera's intrinsic parameters are known, such as image resolution and size L, field of view $\alpha$ (default is 90 degrees horizontal), and camera tilt $\theta$. These allow precise calculation of the angular location of each object with respect to the camera's optical axis.
  11. Step 3: Mapping to World Coordinates in CARLA
  12. The camera's position, rotation, and orientation (height above ground and yaw) are also defined in the simulator. Using these camera configurations and angular relationships, we compute each object’s 3D world position using a pinhole camera model. Under the assumption that vehicles are located on a flat road $z = 0$, the model can be expressed in Equation \ref{['eq2']}.
  13. The transformation accounts for both camera rotation and placement in the co-simulation environment. This approach outputs trajectories in world coordinates, which are used to analyze vehicle behavior near the work zone. A validation process is then followed by comparing the estimated world coordinates (in simulation) with the ground truth trajectories obtained directly from CARLA and SUMO.
  14. Using LiDAR
  15. LiDAR and Camera Data Processing
  16. ...and 14 more sections

Figures (10)

  • Figure 1: Camera-based object detection
  • Figure 2: LiDAR-based Object Detection
  • Figure 3: Co-simulation environments and sensor outputs
  • Figure 4: Ouster LiDAR sensor and an integrated Omnisight radar-camera system at work zone area on US-75
  • Figure 5: Kalman Milter Fused versus single sensor trajectories for multiple vehicles
  • ...and 5 more figures