Personalized Federated Learning for Cross-view Geo-localization

TL;DR

The challenges of data privacy and heterogeneity in autonomous vehicle environments are addressed by proposing a personalized Federated Learning scenario that allows selective sharing of model parameters, and the proposed partial model sharing strategy shows comparable or slightly better performance than classical FL.

Abstract

In this paper we propose a methodology combining Federated Learning (FL) with Cross-view Image Geo-localization (CVGL) techniques. We address the challenges of data privacy and heterogeneity in autonomous vehicle environments by proposing a personalized Federated Learning scenario that allows selective sharing of model parameters. Our method implements a coarse-to-fine approach, where clients share only the coarse feature extractors while keeping fine-grained features specific to local environments. We evaluate our approach against traditional centralized and single-client training schemes using the KITTI dataset combined with satellite imagery. Results demonstrate that our federated CVGL method achieves performance close to centralized training while maintaining data privacy. The proposed partial model sharing strategy shows comparable or slightly better performance than classical FL, offering significant reduced communication overhead without sacrificing accuracy. Our work contributes to more robust and privacy-preserving localization systems for autonomous vehicles operating in diverse environments

Figures (3)

• Figure 1: The client's model is divided into two main components: the front-end and the back-end. The front-end features two parallel VGG-based branches following an autoencoder architecture to extract multi-scale feature maps from satellite and ground view images. The extracted features are then projected to the ground view plane to establish cross-view geometric correspondences. The back-end performs an optimization process using the Levenberg-Marquardt (LM) algorithm, iteratively, for each refining the pose estimation from the coarsest to finer feature levels until convergence or a maximum of five iterations is reached.
• Figure 2: Overview of the Federated Learning Framework that consists of N clients, each using their private datasets to refine a local deep learning-based CVGL model. We consider two scenarios for model exchange: (a) Full Model Sharing, where he entire model is exchanged between clients and (b) Partial Encoder Sharing where the clients share only the encoders of their local model parameters, employing a coarse-to-fine approach.
• Figure 3: Comparison of accuracy across different scenarios and two different metrics. The left image illustrates the lateral accuracy within 5 meters and the right image the azimuth accuracy within five degrees.