COALA: A Practical and Vision-Centric Federated Learning Platform

TL;DR

This paper tackles the gap in practical federated learning for computer vision by introducing COALA, a vision-centric FL platform with a benchmark suite spanning task, data, and model levels. It formalizes a FedAvg-style FL objective and implements diverse CV tasks (15 in total) and data patterns, including semi-supervised, unsupervised, continual, and multi-domain settings, plus flexible model configurations (full, split, multi-model, PEFT). Through systematic experiments on standard datasets such as CIFAR-10/100, DIGITS-5, Office-Caltech, DomainNet, BDD100K, VOC, and MPII, COALA demonstrates the platform's ability to benchmark realistic FL scenarios and reveals important performance gaps and opportunities for robust algorithms under non-IID and evolving data. The three customization axes—configuration, components, and workflow—enable flexible experimentation and practical deployment implications, potentially accelerating CV-FL translation to industry.

Abstract

We present COALA, a vision-centric Federated Learning (FL) platform, and a suite of benchmarks for practical FL scenarios, which we categorize into three levels: task, data, and model. At the task level, COALA extends support from simple classification to 15 computer vision tasks, including object detection, segmentation, pose estimation, and more. It also facilitates federated multiple-task learning, allowing clients to tackle multiple tasks simultaneously. At the data level, COALA goes beyond supervised FL to benchmark both semi-supervised FL and unsupervised FL. It also benchmarks feature distribution shifts other than commonly considered label distribution shifts. In addition to dealing with static data, it supports federated continual learning for continuously changing data in real-world scenarios. At the model level, COALA benchmarks FL with split models and different models in different clients. COALA platform offers three degrees of customization for these practical FL scenarios, including configuration customization, components customization, and workflow customization. We conduct systematic benchmarking experiments for the practical FL scenarios and highlight potential opportunities for further advancements in FL. Codes are open sourced at https://github.com/SonyResearch/COALA.

Figures (9)

• Figure 1: Illustration of three levels of practical FL scenarios supported by COALA. At the task level, we support diverse CV tasks and training of multiple tasks in FL. At the data level, we offer out-of-the-box benchmarks for different types of data heterogeneity, various degrees of data annotation availability, and dynamic changes in data. At the model level, we extend beyond single and full model FL training to split model training and multiple model training with different architectures or parameters on clients.
• Figure 2: Illustration of COALA platform that enables automated benchmarking for practical FL scenarios.
• Figure 3: Domain-wise test accuracy of the global model. The same model does not perform equivalently well on different domains.
• Figure 4: Benchmark of federated semantic segmentation on VOC.
• Figure 5: Federated multiple task learning with 5 tasks and 9 tasks.