Uncertainty-Aware Explainable Federated Learning
Yanci Zhang, Han Yu
TL;DR
This work tackles the gap in explainable federated learning by introducing UncertainXFL, a framework that jointly generates explanations and quantifies their uncertainty within FL. It uses a two-stage approach where clients extract feature-based rules from neural representations and the server aggregates both model updates and rules in a conflict-free, uncertainty-aware manner. Key contributions include a concrete method to compute rule uncertainty from labeller confidence, a greedy uncertainty-guided aggregation strategy, and empirical evidence showing improved explanation and model reliability on image datasets with simulated uncertainty. The approach enhances transparency and robustness in privacy-preserving ML with practical implications for high-stakes domains.
Abstract
Federated Learning (FL) is a collaborative machine learning paradigm for enhancing data privacy preservation. Its privacy-preserving nature complicates the explanation of the decision-making processes and the evaluation of the reliability of the generated explanations. In this paper, we propose the Uncertainty-aware eXplainable Federated Learning (UncertainXFL) to address these challenges. It generates explanations for decision-making processes under FL settings and provides information regarding the uncertainty of these explanations. UncertainXFL is the first framework to explicitly offer uncertainty evaluation for explanations within the FL context. Explanatory information is initially generated by the FL clients and then aggregated by the server in a comprehensive and conflict-free manner during FL training. The quality of the explanations, including the uncertainty score and tested validity, guides the FL training process by prioritizing clients with the most reliable explanations through higher weights during model aggregation. Extensive experimental evaluation results demonstrate that UncertainXFL achieves superior model accuracy and explanation accuracy, surpassing the current state-of-the-art model that does not incorporate uncertainty information by 2.71% and 1.77%, respectively. By integrating and quantifying uncertainty in the data into the explanation process, UncertainXFL not only clearly presents the explanation alongside its uncertainty, but also leverages this uncertainty to guide the FL training process, thereby enhancing the robustness and reliability of the resulting models.