Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CoAst: Validation-Free Contribution Assessment for Federated Learning based on Cross-Round Valuation

Hao Wu, Likun Zhang, Shucheng Li, Fengyuan Xu, Sheng Zhong

TL;DR

CoAst introduces a validation-free contribution assessment framework for federated learning by combining cross-round valuation with layer-wise parameter pruning via ternary quantization. It measures client contributions through the alignment of a client’s round-t parameters with multi-round global updates, using a Signed Cosine-like similarity to emphasize directional agreement. Empirical results on CIFAR-10/100 and STL-10 across multiple model architectures show CoAst matching validation-based methods and outperforming existing validation-free baselines, with robust performance under data quantity, noise, resolution, and masking variations. The approach highlights practical benefits for incentive-compatible FL and scalable contribution analysis, while noting convergence considerations tied to quantization and future improvements.

Abstract

In the federated learning (FL) process, since the data held by each participant is different, it is necessary to figure out which participant has a higher contribution to the model performance. Effective contribution assessment can help motivate data owners to participate in the FL training. Research works in this field can be divided into two directions based on whether a validation dataset is required. Validation-based methods need to use representative validation data to measure the model accuracy, which is difficult to obtain in practical FL scenarios. Existing validation-free methods assess the contribution based on the parameters and gradients of local models and the global model in a single training round, which is easily compromised by the stochasticity of model training. In this work, we propose CoAst, a practical method to assess the FL participants' contribution without access to any validation data. The core idea of CoAst involves two aspects: one is to only count the most important part of model parameters through a weights quantization, and the other is a cross-round valuation based on the similarity between the current local parameters and the global parameter updates in several subsequent communication rounds. Extensive experiments show that CoAst has comparable assessment reliability to existing validation-based methods and outperforms existing validation-free methods.

CoAst: Validation-Free Contribution Assessment for Federated Learning based on Cross-Round Valuation

TL;DR

CoAst introduces a validation-free contribution assessment framework for federated learning by combining cross-round valuation with layer-wise parameter pruning via ternary quantization. It measures client contributions through the alignment of a client’s round-t parameters with multi-round global updates, using a Signed Cosine-like similarity to emphasize directional agreement. Empirical results on CIFAR-10/100 and STL-10 across multiple model architectures show CoAst matching validation-based methods and outperforming existing validation-free baselines, with robust performance under data quantity, noise, resolution, and masking variations. The approach highlights practical benefits for incentive-compatible FL and scalable contribution analysis, while noting convergence considerations tied to quantization and future improvements.

Abstract

In the federated learning (FL) process, since the data held by each participant is different, it is necessary to figure out which participant has a higher contribution to the model performance. Effective contribution assessment can help motivate data owners to participate in the FL training. Research works in this field can be divided into two directions based on whether a validation dataset is required. Validation-based methods need to use representative validation data to measure the model accuracy, which is difficult to obtain in practical FL scenarios. Existing validation-free methods assess the contribution based on the parameters and gradients of local models and the global model in a single training round, which is easily compromised by the stochasticity of model training. In this work, we propose CoAst, a practical method to assess the FL participants' contribution without access to any validation data. The core idea of CoAst involves two aspects: one is to only count the most important part of model parameters through a weights quantization, and the other is a cross-round valuation based on the similarity between the current local parameters and the global parameter updates in several subsequent communication rounds. Extensive experiments show that CoAst has comparable assessment reliability to existing validation-based methods and outperforms existing validation-free methods.
Paper Structure (27 sections, 14 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 27 sections, 14 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Overview
  4. Targeted Scenario
  5. Problem Formalization
  6. CoAst Design
  7. Design Overview
  8. Parameter Pruning
  9. Cross-round Valuation
  10. Implementation
  11. Dataset Settings
  12. Setting 1: Different quantity
  13. Setting 2: Adding noise
  14. Setting 3: Adjusting resolution
  15. Setting 4: Masking
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Contribution assessment in FL scenarios.
  • Figure 2: The CoAst's workflow. Note that the local training procedure is unchanged.
  • Figure 3: Core idea of the cross-round valuation. The $\theta_i^t$, which is consistent with the improvement direction of the model in several rounds, i.e., $U^{(t,k)}$, has a larger contribution.
  • Figure 4: Training samples to demonstrate our dataset settings. Samples are randomly selected from STL-10.