Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune

Mengen Luo, Chi Xu, Ercan Engin Kuruoglu

TL;DR

This work tackles robustness and interpretability in federated learning under data heterogeneity by proposing a trustworthy, personalized Bayesian FL framework with posterior fine-tune (pFedPF). The key idea is to refine approximate client posteriors using normalizing flows, enabling closer alignment to the true posterior while maintaining minimal communication and computation overhead. The approach preserves OOD detection capabilities and improves calibration and predictive reliability, as supported by theoretical analysis and extensive experiments across heterogeneous ID–OOD benchmarks. The results indicate that posterior fine-tuning yields more accurate and trustworthy personalized models that can be readily integrated with existing FL frameworks, enhancing practical applicability in privacy-conscious domains.

Abstract

Performance degradation owing to data heterogeneity and low output interpretability are the most significant challenges faced by federated learning in practical applications. Personalized federated learning diverges from traditional approaches, as it no longer seeks to train a single model, but instead tailors a unique personalized model for each client. However, previous work focused only on personalization from the perspective of neural network parameters and lack of robustness and interpretability. In this work, we establish a novel framework for personalized federated learning, incorporating Bayesian methodology which enhances the algorithm's ability to quantify uncertainty. Furthermore, we introduce normalizing flow to achieve personalization from the parameter posterior perspective and theoretically analyze the impact of normalizing flow on out-of-distribution (OOD) detection for Bayesian neural networks. Finally, we evaluated our approach on heterogeneous datasets, and the experimental results indicate that the new algorithm not only improves accuracy but also outperforms the baseline significantly in OOD detection due to the reliable output of the Bayesian approach.

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune

TL;DR

This work tackles robustness and interpretability in federated learning under data heterogeneity by proposing a trustworthy, personalized Bayesian FL framework with posterior fine-tune (pFedPF). The key idea is to refine approximate client posteriors using normalizing flows, enabling closer alignment to the true posterior while maintaining minimal communication and computation overhead. The approach preserves OOD detection capabilities and improves calibration and predictive reliability, as supported by theoretical analysis and extensive experiments across heterogeneous ID–OOD benchmarks. The results indicate that posterior fine-tuning yields more accurate and trustworthy personalized models that can be readily integrated with existing FL frameworks, enhancing practical applicability in privacy-conscious domains.

Abstract

Performance degradation owing to data heterogeneity and low output interpretability are the most significant challenges faced by federated learning in practical applications. Personalized federated learning diverges from traditional approaches, as it no longer seeks to train a single model, but instead tailors a unique personalized model for each client. However, previous work focused only on personalization from the perspective of neural network parameters and lack of robustness and interpretability. In this work, we establish a novel framework for personalized federated learning, incorporating Bayesian methodology which enhances the algorithm's ability to quantify uncertainty. Furthermore, we introduce normalizing flow to achieve personalization from the parameter posterior perspective and theoretically analyze the impact of normalizing flow on out-of-distribution (OOD) detection for Bayesian neural networks. Finally, we evaluated our approach on heterogeneous datasets, and the experimental results indicate that the new algorithm not only improves accuracy but also outperforms the baseline significantly in OOD detection due to the reliable output of the Bayesian approach.
Paper Structure (25 sections, 5 theorems, 21 equations, 1 figure, 4 tables)

This paper contains 25 sections, 5 theorems, 21 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Preliminaries
  5. Bayesian deep learning
  6. Posterior Of Bayesian Neural Network
  7. Prediction
  8. Normalizing Flow
  9. Radial Flow
  10. Proposed Method
  11. Theoretical Analysis
  12. Out-Of-Distribution Detection Property
  13. Experiments
  14. Setup
  15. Dataset
  16. ...and 10 more sections

Key Result

Theorem 1

Let $\mathbb{R}^d = \cup_{r=1}^{R}Q_r$ and $f|_{Q_r}(x)=U_r x + c_r$ be the affine representation in piecewise representation of the output of a ReLU network on $Q_r$. For almost any $x \in \mathbb{R}^n$ and $\forall \epsilon > 0$, there $\exists \delta > 0$ and class $i \in{1,...,k}$ such that $\te

Figures (1)

  • Figure 1: The reliability diagram of all algorithm in CIFAR10 dataset. This signifies that our approach will rectify the issue of model overconfidence.

Theorems & Definitions (5)

  • Theorem 1: ReLU Overconfident
  • Theorem 2: Approximation With NF
  • Lemma 1
  • Lemma 2
  • Lemma 3