Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedPylot: Navigating Federated Learning for Real-Time Object Detection in Internet of Vehicles

Cyprien Quéméneur, Soumaya Cherkaoui

TL;DR

This paper addresses the challenge of privacy-preserving, real-time object detection in the Internet of Vehicles by federating training of a modern detector (YOLOv7) across distributed edge-like clients on HPC infrastructure. It introduces FedPylot, an MPI-based prototype with a hybrid encryption scheme, enabling controlled, synchronous federated optimization (primarily using FedAvgM/FedOpt) and evaluating performance under realistic non-IID data shifts, including unbalancedness and long-tail label distributions. The work provides a thorough experimental study on KITTI and nuImages, showing that server-side momentum and carefully tuned learning rates can improve federated accuracy (e.g., KITTI mAP from 57.5% to 66.3%), while highlighting that excessive momentum or severe long-tail skew can hinder convergence. The results demonstrate that FL can approach centralized performance for real-time detection, quantify the trade-offs in communication and latency, and offer an open-source platform to spur further research in federated, real-time automotive perception. FedPylot thus offers a practical, scalable foundation for future privacy-aware IoV deployments and broadens the experimental space for federated object detection on large-scale HPC systems.

Abstract

The Internet of Vehicles (IoV) emerges as a pivotal component for autonomous driving and intelligent transportation systems (ITS), by enabling low-latency big data processing in a dense interconnected network that comprises vehicles, infrastructures, pedestrians and the cloud. Autonomous vehicles are heavily reliant on machine learning (ML) and can strongly benefit from the wealth of sensory data generated at the edge, which calls for measures to reconcile model training with preserving the privacy of sensitive user data. Federated learning (FL) stands out as a promising solution to train sophisticated ML models in vehicular networks while protecting the privacy of road users and mitigating communication overhead. This paper examines the federated optimization of the cutting-edge YOLOv7 model to tackle real-time object detection amid data heterogeneity, encompassing unbalancedness, concept drift, and label distribution skews. To this end, we introduce FedPylot, a lightweight MPI-based prototype to simulate federated object detection experiments on high-performance computing (HPC) systems, where we safeguard server-client communications using hybrid encryption. Our study factors in accuracy, communication cost, and inference speed, thereby presenting a balanced approach to the challenges faced by autonomous vehicles. We demonstrate promising results for the applicability of FL in IoV and hope that FedPylot will provide a basis for future research into federated real-time object detection. The source code is available at https://github.com/cyprienquemeneur/fedpylot.

FedPylot: Navigating Federated Learning for Real-Time Object Detection in Internet of Vehicles

TL;DR

This paper addresses the challenge of privacy-preserving, real-time object detection in the Internet of Vehicles by federating training of a modern detector (YOLOv7) across distributed edge-like clients on HPC infrastructure. It introduces FedPylot, an MPI-based prototype with a hybrid encryption scheme, enabling controlled, synchronous federated optimization (primarily using FedAvgM/FedOpt) and evaluating performance under realistic non-IID data shifts, including unbalancedness and long-tail label distributions. The work provides a thorough experimental study on KITTI and nuImages, showing that server-side momentum and carefully tuned learning rates can improve federated accuracy (e.g., KITTI mAP from 57.5% to 66.3%), while highlighting that excessive momentum or severe long-tail skew can hinder convergence. The results demonstrate that FL can approach centralized performance for real-time detection, quantify the trade-offs in communication and latency, and offer an open-source platform to spur further research in federated, real-time automotive perception. FedPylot thus offers a practical, scalable foundation for future privacy-aware IoV deployments and broadens the experimental space for federated object detection on large-scale HPC systems.

Abstract

The Internet of Vehicles (IoV) emerges as a pivotal component for autonomous driving and intelligent transportation systems (ITS), by enabling low-latency big data processing in a dense interconnected network that comprises vehicles, infrastructures, pedestrians and the cloud. Autonomous vehicles are heavily reliant on machine learning (ML) and can strongly benefit from the wealth of sensory data generated at the edge, which calls for measures to reconcile model training with preserving the privacy of sensitive user data. Federated learning (FL) stands out as a promising solution to train sophisticated ML models in vehicular networks while protecting the privacy of road users and mitigating communication overhead. This paper examines the federated optimization of the cutting-edge YOLOv7 model to tackle real-time object detection amid data heterogeneity, encompassing unbalancedness, concept drift, and label distribution skews. To this end, we introduce FedPylot, a lightweight MPI-based prototype to simulate federated object detection experiments on high-performance computing (HPC) systems, where we safeguard server-client communications using hybrid encryption. Our study factors in accuracy, communication cost, and inference speed, thereby presenting a balanced approach to the challenges faced by autonomous vehicles. We demonstrate promising results for the applicability of FL in IoV and hope that FedPylot will provide a basis for future research into federated real-time object detection. The source code is available at https://github.com/cyprienquemeneur/fedpylot.
Paper Structure (22 sections, 6 equations, 9 figures, 2 tables, 1 algorithm)

This paper contains 22 sections, 6 equations, 9 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Federated learning
  4. Formulation
  5. Data heterogeneity
  6. Other challenges in FL
  7. Object detection
  8. YOLOv7
  9. Related work
  10. System design
  11. Experiments
  12. Prototype implementation details
  13. Federated simulation
  14. Encryption
  15. Datasets
  16. ...and 7 more sections

Figures (9)

  • Figure 1: Vehicular clients collaboratively learn a joint model with FL. The vehicles collect driving data using various sensors to train their own local model, while a central server is responsible for regularly gathering and aggregating local weight-update vectors to compute a global model, which is subsequently disseminated to the clients for further training. The raw data are kept private, but are typically non-identically distributed due to the decentralized nature of the clients (samples taken from nuImages).
  • Figure 2: Hybrid cryptosystem for server-client communications. Transmissions between the server and the vehicular clients are encrypted using a highly efficient symmetric algorithm. The symmetric key is generated by the server and protected using a public-key cryptosystem when passed to the clients.
  • Figure 3: Distribution of samples in nuImages training data. The dataset is composed of nearly 500 driving logs acquired through six different cameras, and features a wide range of driving scenarios and weather conditions
  • Figure 4: KITTI split. 25% of KITTI training data are stored on the server, while the 75% left are distributed IID among five clients. We report (a) the number of samples held by each client, (b) the normalized number of annotations held per sample per client, and (c) the label distribution in the original training set.
  • Figure 5: nuImages-10 split. The original classes are mapped to ten labels, and the training data are split non-IID among ten clients. We report (a) the number of samples held by each client, (b) the normalized number of annotations held per sample per client, and (c) the label distribution in the original training set.
  • ...and 4 more figures