Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Vehicle Environmental Awareness via Federated Learning and Automatic Labeling

Chih-Yu Lin, Jin-Wei Liang

TL;DR

This work tackles vehicle identification (VID) by fusing front-view image data with vehicle-to-vehicle messages under privacy constraints. It introduces FedMDFNN, a federated MDFNN framework that uses automatic labeling based on license plate recognition and data augmentation to generate labeled data without driver input, while keeping raw data on-device. Experiments in CARLA show that auto-labeling plus augmentation yields substantial labeling coverage (roughly $24\%$ per image), and BBX-MDFNN with auto-labeled data can reach a high VID accuracy (CR_total around $83.56\%$) close to manually labeled data (CR_total $85.63\%$), with some degradation under federated settings. Overall, the approach demonstrates a practical, privacy-preserving path toward enhanced vehicle environmental awareness, with future work addressing non-IID data and integrating object tracking for richer training data.

Abstract

Vehicle environmental awareness is a crucial issue in improving road safety. Through a variety of sensors and vehicle-to-vehicle communication, vehicles can collect a wealth of data. However, to make these data useful, sensor data must be integrated effectively. This paper focuses on the integration of image data and vehicle-to-vehicle communication data. More specifically, our goal is to identify the locations of vehicles sending messages within images, a challenge termed the vehicle identification problem. In this paper, we employ a supervised learning model to tackle the vehicle identification problem. However, we face two practical issues: first, drivers are typically unwilling to share privacy-sensitive image data, and second, drivers usually do not engage in data labeling. To address these challenges, this paper introduces a comprehensive solution to the vehicle identification problem, which leverages federated learning and automatic labeling techniques in combination with the aforementioned supervised learning model. We have validated the feasibility of our proposed approach through experiments.

Enhancing Vehicle Environmental Awareness via Federated Learning and Automatic Labeling

TL;DR

This work tackles vehicle identification (VID) by fusing front-view image data with vehicle-to-vehicle messages under privacy constraints. It introduces FedMDFNN, a federated MDFNN framework that uses automatic labeling based on license plate recognition and data augmentation to generate labeled data without driver input, while keeping raw data on-device. Experiments in CARLA show that auto-labeling plus augmentation yields substantial labeling coverage (roughly per image), and BBX-MDFNN with auto-labeled data can reach a high VID accuracy (CR_total around ) close to manually labeled data (CR_total ), with some degradation under federated settings. Overall, the approach demonstrates a practical, privacy-preserving path toward enhanced vehicle environmental awareness, with future work addressing non-IID data and integrating object tracking for richer training data.

Abstract

Vehicle environmental awareness is a crucial issue in improving road safety. Through a variety of sensors and vehicle-to-vehicle communication, vehicles can collect a wealth of data. However, to make these data useful, sensor data must be integrated effectively. This paper focuses on the integration of image data and vehicle-to-vehicle communication data. More specifically, our goal is to identify the locations of vehicles sending messages within images, a challenge termed the vehicle identification problem. In this paper, we employ a supervised learning model to tackle the vehicle identification problem. However, we face two practical issues: first, drivers are typically unwilling to share privacy-sensitive image data, and second, drivers usually do not engage in data labeling. To address these challenges, this paper introduces a comprehensive solution to the vehicle identification problem, which leverages federated learning and automatic labeling techniques in combination with the aforementioned supervised learning model. We have validated the feasibility of our proposed approach through experiments.
Paper Structure (21 sections, 7 equations, 10 figures, 7 tables, 2 algorithms)

This paper contains 21 sections, 7 equations, 10 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vehicle environmental awareness
  4. Federated Learning
  5. Automatic Labeling
  6. FedMDFNN
  7. Data Collection Module
  8. Auto Labeling and Data Augmentation Module
  9. Auto Labeling Submodule
  10. Data Augmentation Submodule
  11. Vehicle Mapping Module
  12. Data Preprocessing Submodule
  13. MDFNN Submodule
  14. Mapping Decision Submodule
  15. Federated Learning based MDFNN (FedMDFNN)
  16. ...and 6 more sections

Figures (10)

  • Figure 1: The vehicle identification (VID) problem.
  • Figure 2: System architecture of FedMDFNN.
  • Figure 3: Examples of OCR failure.
  • Figure 4: Built-in license plates in the Carla simulatorcarla_Dosovitskiy17.
  • Figure 5: The front, rear, and sides of the vehicle. $P_{auto}^t$ is acquired through the automatic labeling submodule, while $V_{Rear}^t$ and $V_{Outside}^t$ are obtained via the data augmentation submodule.
  • ...and 5 more figures