When Cars meet Drones: Hyperbolic Federated Learning for Source-Free Domain Adaptation in Adverse Weather

TL;DR

A novel approach is proposed which exploits a batch-norm weather-aware strategy to dynamically adapt the model to the different weather conditions, while hyperbolic space prototypes are used to align the heterogeneous client representations.

Abstract

In Federated Learning (FL), multiple clients collaboratively train a global model without sharing private data. In semantic segmentation, the Federated source Free Domain Adaptation (FFreeDA) setting is of particular interest, where clients undergo unsupervised training after supervised pretraining at the server side. While few recent works address FL for autonomous vehicles, intrinsic real-world challenges such as the presence of adverse weather conditions and the existence of different autonomous agents are still unexplored. To bridge this gap, we address both problems and introduce a new federated semantic segmentation setting where both car and drone clients co-exist and collaborate. Specifically, we propose a novel approach for this setting which exploits a batch-norm weather-aware strategy to dynamically adapt the model to the different weather conditions, while hyperbolic space prototypes are used to align the heterogeneous client representations. Finally, we introduce FLYAWARE, the first semantic segmentation dataset with adverse weather data for aerial vehicles.

Figures (12)

• Figure 1: Federated Learning for autonomous driving across diverse conditions: each client trains the segmentation model locally with its own data, which may be biased towards specific environmental conditions. E.g., a taxi operating predominantly at night will have most training samples in nighttime conditions. Despite the local biases, the approach aims to create a single, robust model capable of generalizing across diverse driving scenarios.
• Figure 2: The two-step training process of HyperFLAW: server pretraining on synthetic data and client-side real-world adaptation. We highlight the active modeling of weather conditions using weather-aware batch norm layers and the use of hyperbolic space for feature alignment via prototypical learning, ensuring consistency in features extracted by car and drone agents across various atmospheric conditions. At the server, the aggregation with the queue of previous global models reduces the instability introduced by aggregating unlabeled clients.
• Figure 3: Comparison of Weather-Batch Normalization with BN personalization methods li2021fedbnandreux2020siloed.
• Figure 4: FLYAWARE: samples under different weathers.
• Figure 5: Euclidean (dashed lines) vs hyperbolic (solid lines) prototypes.