Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Influence-oriented Personalized Federated Learning

Yue Tan, Guodong Long, Jing Jiang, Chengqi Zhang

TL;DR

An influence-oriented federated learning framework, namely FedC^2I, which quantitatively measures Client-level and Class-level Influence to realize adaptive parameter aggregation for each client and evaluates the performance of the method against existing federated learning methods under non-IID settings.

Abstract

Traditional federated learning (FL) methods often rely on fixed weighting for parameter aggregation, neglecting the mutual influence by others. Hence, their effectiveness in heterogeneous data contexts is limited. To address this problem, we propose an influence-oriented federated learning framework, namely FedC^2I, which quantitatively measures Client-level and Class-level Influence to realize adaptive parameter aggregation for each client. Our core idea is to explicitly model the inter-client influence within an FL system via the well-crafted influence vector and influence matrix. The influence vector quantifies client-level influence, enables clients to selectively acquire knowledge from others, and guides the aggregation of feature representation layers. Meanwhile, the influence matrix captures class-level influence in a more fine-grained manner to achieve personalized classifier aggregation. We evaluate the performance of FedC^2I against existing federated learning methods under non-IID settings and the results demonstrate the superiority of our method.

Influence-oriented Personalized Federated Learning

TL;DR

An influence-oriented federated learning framework, namely FedC^2I, which quantitatively measures Client-level and Class-level Influence to realize adaptive parameter aggregation for each client and evaluates the performance of the method against existing federated learning methods under non-IID settings.

Abstract

Traditional federated learning (FL) methods often rely on fixed weighting for parameter aggregation, neglecting the mutual influence by others. Hence, their effectiveness in heterogeneous data contexts is limited. To address this problem, we propose an influence-oriented federated learning framework, namely FedC^2I, which quantitatively measures Client-level and Class-level Influence to realize adaptive parameter aggregation for each client. Our core idea is to explicitly model the inter-client influence within an FL system via the well-crafted influence vector and influence matrix. The influence vector quantifies client-level influence, enables clients to selectively acquire knowledge from others, and guides the aggregation of feature representation layers. Meanwhile, the influence matrix captures class-level influence in a more fine-grained manner to achieve personalized classifier aggregation. We evaluate the performance of FedC^2I against existing federated learning methods under non-IID settings and the results demonstrate the superiority of our method.
Paper Structure (28 sections, 10 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 28 sections, 10 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Influence-Oriented Deep Learning
  5. Problem Formulation
  6. FedC$^{2}$I: An Influence-Oriented Federated Learning Framework
  7. An Overview of FedC$^2$I
  8. Client-Level Influence Measure
  9. Class-Level Influence Measure
  10. Client Update
  11. Experiments
  12. Experimental Setup
  13. Datasets.
  14. Baselines.
  15. Implementation Details.
  16. ...and 13 more sections

Figures (4)

  • Figure 1: (a) Five clients owning digit images from various domains. Some share the same background color, font style, and/or hand-writing habit, forming an underlying correlation among clients. (b) Different understandings of digit "7" held by different clients.
  • Figure 2: (a) An overview of the influence-oriented aggregation procedure in our proposed FedC$^2$I. Green boxes and orange boxes correspond to the feature representation layers and classifiers, respectively. The loss vector is comprised of loss values. Blue boxes correspond to the models that are trained locally and personalized for each client. Orange boxes correspond to the models aggregated at the server, with knowledge shared across clients. (b) An illustration of the data distribution and model aggregation scheme.
  • Figure 3: Visualization of influence vector in different communication rounds. (a)-(d) The value of influence vector in round 1, 2, 10, and 20 on Digit-5 dataset. (e)-(h) The value of influence vector in round 1, 4, 10, and 40 on Office-10 dataset. Each figure shows the client-level influence values over all clients.
  • Figure 4: Examples of raw instances from two datasets: Digit-5 (left) and Office-10 (right). We present five classes for each dataset to show the feature shift across their sub-datasets.

Theorems & Definitions (2)

  • Definition 4.1
  • Definition 4.2