Exploring Federated Learning for Thermal Urban Feature Segmentation -- A Comparison of Centralized and Decentralized Approaches

TL;DR

This work evaluates Federated Learning for UAV-based thermal urban feature segmentation using real-world, non-IID data from two German cities. It implements NVFlare-based FL with multiple aggregation algorithms (FedAvg, FedProx, FedOpt, Scaffold) and workflows (centralized, decentralized, Swarm Learning, cyclic weight transfer) and compares them to centralized learning on segmentation accuracy, training time, and energy use. Key findings show FL can achieve comparable accuracy to CL, with FedAvg and Scaffold often performing best depending on data distribution; BN normalization issues are mitigated by Group Normalization, and decentralized workflows like Swarm Learning can substantially reduce training time at some costs. The study provides practical guidance on selecting FL strategies for UAV thermal segmentation, highlighting the trade-offs among accuracy, convergence speed, and energy efficiency in real-world deployments.

Abstract

Federated Learning (FL) is an approach for training a shared Machine Learning (ML) model with distributed training data and multiple participants. FL allows bypassing limitations of the traditional Centralized Machine Learning CL if data cannot be shared or stored centrally due to privacy or technical restrictions -- the participants train the model locally with their training data and do not need to share it among the other participants. This paper investigates the practical implementation and effectiveness of FL in a real-world scenario, specifically focusing on unmanned aerial vehicle (UAV)-based thermal images for common thermal feature detection in urban environments. The distributed nature of the data arises naturally and makes it suitable for FL applications, as images captured in two German cities are available. This application presents unique challenges due to non-identical distribution and feature characteristics of data captured at both locations. The study makes several key contributions by evaluating FL algorithms in real deployment scenarios rather than simulation. We compare several FL approaches with a centralized learning baseline across key performance metrics such as model accuracy, training time, communication overhead, and energy usage. This paper also explores various FL workflows, comparing client-controlled workflows and server-controlled workflows. The findings of this work serve as a valuable reference for understanding the practical application and limitations of the FL methods in segmentation tasks in UAV-based imaging.

Figures (3)

• Figure 1: Multispectral (RGB/TIR) image and annotations example 10858702.
• Figure 2: Energy consumption and runtime by each site and each algorithm.
• Figure 3: Energy consumption and runtime by each site and each workflow.