Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving

TL;DR

The paper tackles the challenge of generalizing end-to-end autonomous driving under constrained communications by introducing CRCHFL, an optimization-driven, hierarchical federated learning framework that couples cloud pretraining, edge FL, and cloud FL. By jointly scheduling data and model transfers within a throughput budget using an objective that balances pretraining data and federated updates, CRCHFL accelerates convergence and improves generalization compared to SFL and HFL, as demonstrated in CARLA simulations. Key contributions include (1) an optimization-based resource scheduler, (2) a three-stage training workflow that leverages centralized pretraining to seed edge-cloud learning, and (3) empirical evidence of faster convergence and better performance under tight bandwidth, with ablation insights on resource allocation. The approach has practical implications for deploying reliable, generalizable end-to-end autonomous driving in resource-limited networks, while future work could address channel fading, larger-scale deployments, and multi-modal CRCHFL integrations.

Abstract

While federated learning (FL) improves the generalization of end-to-end autonomous driving by model aggregation, the conventional single-hop FL (SFL) suffers from slow convergence rate due to long-range communications among vehicles and cloud server. Hierarchical federated learning (HFL) overcomes such drawbacks via introduction of mid-point edge servers. However, the orchestration between constrained communication resources and HFL performance becomes an urgent problem. This paper proposes an optimization-based Communication Resource Constrained Hierarchical Federated Learning (CRCHFL) framework to minimize the generalization error of the autonomous driving model using hybrid data and model aggregation. The effectiveness of the proposed CRCHFL is evaluated in the Car Learning to Act (CARLA) simulation platform. Results show that the proposed CRCHFL both accelerates the convergence rate and enhances the generalization of federated learning autonomous driving model. Moreover, under the same communication resource budget, it outperforms the HFL by 10.33% and the SFL by 12.44%.

Figures (8)

• Figure 1: Illustration of a cloud-edge-vehicle system. Red bars represent wireless flows. Blue bars represent wireline flows. The size of each bar represents the communication throughput of its associated link.
• Figure 2: The structure of the proposed CRCHFL. The first layer (top layer) is overview of imitation learning pipeline, including input sensor data, imitation learning DNN model and predicted output driving actions. Second layer showcases the CRCHFL-involved nodes, i.e., autonomous vehicles, edge servers and cloud server. The third layer is about the communication of CRCHFL, including data and model upload (UL) and download (DL). The fourth layer (bottom layer) focuses on optimization algorithm to schedule communication resources.
• Figure 3: Illustration of entire training and inference process of autonomous driving vehicle. $\mathbf{Digitizer}$ is used to quantize the steer action into 7-level digital signal, while $\mathbf{DAC}$ is utilized to convert the predicted digitized steer to an analogical steer to drive the ego-vehicle. The proposed $\mathbf{Model}$ contains two mutually independent branches where $\mathbf{Branch\ I}$ is responsible for brake and throttle signal and $\mathbf{Branch\ II}$ is used to predict the steer signal.
• Figure 4: (a) Comparison of evaluation Accuracy of SFL, HFL and CRCHFL w.r.t FL rounds. (b) Comparison of evaluation Loss of SFL, HFL and CRCHFL w.r.t FL rounds. These experiments are all conducted under 20GB throughput budget.
• Figure 5: (a) Comparison of evaluation Accuracy of SFL, HFL and CRCHFL w.r.t consumed throughput in one training process. (b) Comparison of evaluation Loss of SFL, HFL and CRCHFL w.r.t consumed throughput in one training process. These experiments are all conducted under 20GB throughput budget.