Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentation

TL;DR

The paper tackles data islands in remote sensing semantic segmentation by introducing GeoFed, a geographic heterogeneity-aware federated learning framework. GeoFed combines Global Insight Enhancement, Essential Feature Mining, and Local-Global Balance to address class-distribution and object-appearance heterogeneity while preserving privacy. Across three re-organized RSS datasets, GeoFed achieves state-of-the-art or competitive results, with ablations confirming the separate and joint benefits of GIE, EFM, and LoGo. The work advances privacy-preserving cross-institution RSS collaboration and highlights potential extensions to other RSS interpretation tasks and heterogeneous-model settings.

Abstract

Remote sensing semantic segmentation (RSS) is an essential technology in earth observation missions. Due to concerns over geographic information security, data privacy, storage bottleneck and industry competition, high-quality annotated remote sensing images are often isolated and distributed across institutions. The issue of remote sensing data islands poses challenges for fully utilizing isolated datasets to train a global model. Federated learning (FL), a privacy-preserving distributed collaborative learning technology, offers a potential solution to leverage isolated remote sensing data. Typically, remote sensing images from different institutions exhibit significant geographic heterogeneity, characterized by coupled class-distribution heterogeneity and object-appearance heterogeneity. However, existing FL methods lack consideration of them, leading to a decline in the performance of the global model when FL is directly applied to RSS. We propose a novel Geographic heterogeneity-aware Federated learning (GeoFed) framework to bridge data islands in RSS. Our framework consists of three modules, including the Global Insight Enhancement (GIE) module, the Essential Feature Mining (EFM) module and the Local-Global Balance (LoGo) module. Through the GIE module, class distribution heterogeneity is alleviated by introducing a prior global class distribution vector. We design an EFM module to alleviate object appearance heterogeneity by constructing essential features. Furthermore, the LoGo module enables the model to possess both global generalization capability and local adaptation. Extensive experiments on three public datasets (i.e., FedFBP, FedCASID, FedInria) demonstrate that our GeoFed framework consistently outperforms the current state-of-the-art methods.

Figures (12)

• Figure 1: (a) Illustration of bridging remote sensing data islands. Institutions only transmit model parameters without disclosing their private data, therefore achieving privacy-preserving collaborative learning. (b) Traditional FL applied in remote sensing semantic segmentation encounters geographic heterogeneity. The coexistence of class distribution heterogeneity and object appearance heterogeneity limits the model performance.
• Figure 2: (a)Illustration of the traditional remote sensing semantic segmentation training paradigm. ① All collaborating institutions upload their private remote sensing data to the server for centralized storage. ② The server conducts centralized training. ③ The server distributes the trained models to the respective institutions. (b)Illustration of the FL paradigm for remote sensing semantic segmentation. ① Distribution of the global model. ② Update of local models. ③ Upload of local models. ④ global models aggregation.
• Figure 3: An overview of our proposed GeoFed framework. It mainly contains three components. Firstly, (a) class-distribution heterogeneity is alleviated through the utilization of the GIE module. GIE expands the feature diversity of local models under the global class distribution and injects the global information. Next, (b) an EFM module containing intra & inter contrastive loss is applied to alleviate object-appearance heterogeneity. Lastly, (c) a LoGo module with three branches is applied to achieve the balance between local characteristics and global generalization.
• Figure 4: The MPC process schematic diagram.$R$ represents a relatively large private random number generated by the first institution, and $P_i$ denotes the local private class distribution vectors of the i-th institution. Institutions sequentially add their local class distribution vectors in this process and pass the results to the next institution. The last institution directly transmits the result to the first institution. After subtracting R, the first institution broadcasts the global class distribution to all participants.
• Figure 5: Illustration of Essential Feature Mining.