Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review

TL;DR

This scoping review investigates how Federated Learning (FL) and Explainable AI (XAI) interact, focusing on whether FL affects model explanations and how explanations influence FL training. Analyzing 37 studies from 2019–2023, it finds only one study that quantitatively assessed FL’s impact on explanations, revealing a major research gap. The work categorizes interplay into four facets—FL impacting explanations, explanations impacting FL training, FL impacting interpretability, and interpretability impacting FL training—and notes a reliance on post-hoc explanations with limited use of established FL libraries and inconsistent reporting. It calls for more rigorous, transparent methodologies and the development of federated-specific local explanations to support trustworthy, privacy-preserving AI in high-stakes domains such as healthcare and finance.

Abstract

The joint implementation of federated learning (FL) and explainable artificial intelligence (XAI) could allow training models from distributed data and explaining their inner workings while preserving essential aspects of privacy. Toward establishing the benefits and tensions associated with their interplay, this scoping review maps the publications that jointly deal with FL and XAI, focusing on publications that reported an interplay between FL and model interpretability or post-hoc explanations. Out of the 37 studies meeting our criteria, only one explicitly and quantitatively analyzed the influence of FL on model explanations, revealing a significant research gap. The aggregation of interpretability metrics across FL nodes created generalized global insights at the expense of node-specific patterns being diluted. Several studies proposed FL algorithms incorporating explanation methods to safeguard the learning process against defaulting or malicious nodes. Studies using established FL libraries or following reporting guidelines are a minority. More quantitative research and structured, transparent practices are needed to fully understand their mutual impact and under which conditions it happens.

Figures (10)

• Figure 1: Schematic overview of a main research question: do explanations differ between federated models and centralized models?
• Figure 2: Flow diagram indicating the number of papers identified, excluded in the different phases, and included in the review.
• Figure 3: Distribution of reviewed articles across different XAI techniques. Explanation methods (shades of blue) were studied by 64.8% of the included works, whereas interpretable models (shades of green) were studied by 35.1% of the included works.
• Figure 4: Venn diagram showing the number of studies using each type of FL. Vertical and transfer FL are used far less than horizontal FL.
• Figure 5: Distribution of reviewed studies according to whether a FL library or an own implementation is used. Less than a third of the reviewed papers used established papers, and about a third did not specify how they implemented FL.