Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learn How to Query from Unlabeled Data Streams in Federated Learning

Yuchang Sun, Xinran Li, Tao Lin, Jun Zhang

TL;DR

This work addresses the challenge of training in federated learning when unlabeled data arrive as streams and labeling is costly. It introduces LeaDQ, a multi-agent reinforcement learning framework that learns decentralized data-querying policies under centralized training and decentralized execution (CTDE), coordinating clients via a QMIX-based mixer to align local sample selections with global model objectives. By formulating data querying as a Dec-POMDP and using a shared held-out reward, LeaDQ enables cooperative data selection without data sharing, improving global model accuracy across image and text tasks in non-IID FL settings. The results show LeaDQ outperforms representative baselines and demonstrates robustness to varying data arrival rates and heterogeneity, with practical implications for privacy-preserving, label-efficient FL in streaming environments.

Abstract

Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data. Existing studies on FL typically assume offline labeled data available at each client when the training starts. Nevertheless, the training data in practice often arrive at clients in a streaming fashion without ground-truth labels. Given the expensive annotation cost, it is critical to identify a subset of informative samples for labeling on clients. However, selecting samples locally while accommodating the global training objective presents a challenge unique to FL. In this work, we tackle this conundrum by framing the data querying process in FL as a collaborative decentralized decision-making problem and proposing an effective solution named LeaDQ, which leverages multi-agent reinforcement learning algorithms. In particular, under the implicit guidance from global information, LeaDQ effectively learns the local policies for distributed clients and steers them towards selecting samples that can enhance the global model's accuracy. Extensive simulations on image and text tasks show that LeaDQ advances the model performance in various FL scenarios, outperforming the benchmarking algorithms.

Learn How to Query from Unlabeled Data Streams in Federated Learning

TL;DR

This work addresses the challenge of training in federated learning when unlabeled data arrive as streams and labeling is costly. It introduces LeaDQ, a multi-agent reinforcement learning framework that learns decentralized data-querying policies under centralized training and decentralized execution (CTDE), coordinating clients via a QMIX-based mixer to align local sample selections with global model objectives. By formulating data querying as a Dec-POMDP and using a shared held-out reward, LeaDQ enables cooperative data selection without data sharing, improving global model accuracy across image and text tasks in non-IID FL settings. The results show LeaDQ outperforms representative baselines and demonstrates robustness to varying data arrival rates and heterogeneity, with practical implications for privacy-preserving, label-efficient FL in streaming environments.

Abstract

Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data. Existing studies on FL typically assume offline labeled data available at each client when the training starts. Nevertheless, the training data in practice often arrive at clients in a streaming fashion without ground-truth labels. Given the expensive annotation cost, it is critical to identify a subset of informative samples for labeling on clients. However, selecting samples locally while accommodating the global training objective presents a challenge unique to FL. In this work, we tackle this conundrum by framing the data querying process in FL as a collaborative decentralized decision-making problem and proposing an effective solution named LeaDQ, which leverages multi-agent reinforcement learning algorithms. In particular, under the implicit guidance from global information, LeaDQ effectively learns the local policies for distributed clients and steers them towards selecting samples that can enhance the global model's accuracy. Extensive simulations on image and text tasks show that LeaDQ advances the model performance in various FL scenarios, outperforming the benchmarking algorithms.

Paper Structure

This paper contains 38 sections, 16 equations, 5 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Active learning
  4. Federated active learning
  5. Problem Formulation
  6. Active data querying
  7. Federated model training
  8. Motivating Example
  9. LeaDQ: Learning the Data Querying Policies for Clients
  10. Data Querying in FL as A Decentralized Decision-Making Process
  11. Decentralized Policy Execution for Querying Data Samples
  12. Centralized Policy Training for Learning Local Policies
  13. Experiments
  14. Setup
  15. Baselines
  16. ...and 23 more sections

Figures (5)

  • Figure 1: Workflow of the proposed LeaDQ framework.
  • Figure 2: An overview of the proposed LeaDQ framework. The yellow block is the process of active data querying and the blue block is the process of federated model training. The dashed lines are conducted only when training the query policies.
  • Figure 3: Comparisons between different data querying strategies. In (a), the model accuracy is computed on the test data; in (b), the distribution divergence is computed as the Kullback–Leibler (KL) divergence between the distributions of selected unlabeled data on all clients and the target data.
  • Figure 4: Model accuracy (%) in different data querying rounds.
  • Figure 5: Visualization of the data samples in SVHN dataset ($R=100$). (a) Label distribution of local data on a random client, target global data, and queried data by LeaDQ; (b) T-SNE plots tsne of feature distributions of the global data and queried data by LeaDQ.