Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Interpretable Client Decision Tree Aggregation process for Federated Learning

Alberto Argente-Garrido, Cristina Zuheros, M. Victoria Luzón, Francisco Herrera

TL;DR

This work tackles the challenge of merging self-explanatory decision trees in Federated Learning without sacrificing interpretability or privacy. It introduces ICDTA4FL, a single-round, tree-agnostic aggregation framework that combines local decision paths into a global DT, supporting both ID3 and CART. By filtering out low-quality trees and merging rules across clients, ICDTA4FL achieves superior or competitive performance compared to state-of-the-art ID3 approaches across IID and non-IID data from four datasets, while preserving the interpretability of the resulting model. The approach is communication-efficient and scalable to many clients, with demonstrated robustness to data heterogeneity and a clear emphasis on explainability for trustworthy AI in distributed settings.

Abstract

Trustworthy Artificial Intelligence solutions are essential in today's data-driven applications, prioritizing principles such as robustness, safety, transparency, explainability, and privacy among others. This has led to the emergence of Federated Learning as a solution for privacy and distributed machine learning. While decision trees, as self-explanatory models, are ideal for collaborative model training across multiple devices in resource-constrained environments such as federated learning environments for injecting interpretability in these models. Decision tree structure makes the aggregation in a federated learning environment not trivial. They require techniques that can merge their decision paths without introducing bias or overfitting while keeping the aggregated decision trees robust and generalizable. In this paper, we propose an Interpretable Client Decision Tree Aggregation process for Federated Learning scenarios that keeps the interpretability and the precision of the base decision trees used for the aggregation. This model is based on aggregating multiple decision paths of the decision trees and can be used on different decision tree types, such as ID3 and CART. We carry out the experiments within four datasets, and the analysis shows that the tree built with the model improves the local models, and outperforms the state-of-the-art.

An Interpretable Client Decision Tree Aggregation process for Federated Learning

TL;DR

This work tackles the challenge of merging self-explanatory decision trees in Federated Learning without sacrificing interpretability or privacy. It introduces ICDTA4FL, a single-round, tree-agnostic aggregation framework that combines local decision paths into a global DT, supporting both ID3 and CART. By filtering out low-quality trees and merging rules across clients, ICDTA4FL achieves superior or competitive performance compared to state-of-the-art ID3 approaches across IID and non-IID data from four datasets, while preserving the interpretability of the resulting model. The approach is communication-efficient and scalable to many clients, with demonstrated robustness to data heterogeneity and a clear emphasis on explainability for trustworthy AI in distributed settings.

Abstract

Trustworthy Artificial Intelligence solutions are essential in today's data-driven applications, prioritizing principles such as robustness, safety, transparency, explainability, and privacy among others. This has led to the emergence of Federated Learning as a solution for privacy and distributed machine learning. While decision trees, as self-explanatory models, are ideal for collaborative model training across multiple devices in resource-constrained environments such as federated learning environments for injecting interpretability in these models. Decision tree structure makes the aggregation in a federated learning environment not trivial. They require techniques that can merge their decision paths without introducing bias or overfitting while keeping the aggregated decision trees robust and generalizable. In this paper, we propose an Interpretable Client Decision Tree Aggregation process for Federated Learning scenarios that keeps the interpretability and the precision of the base decision trees used for the aggregation. This model is based on aggregating multiple decision paths of the decision trees and can be used on different decision tree types, such as ID3 and CART. We carry out the experiments within four datasets, and the analysis shows that the tree built with the model improves the local models, and outperforms the state-of-the-art.
Paper Structure (30 sections, 2 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 30 sections, 2 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Federated Learning
  4. Decision Trees and Interpretability
  5. Decision Trees in Federated Learning
  6. An Interpretable Client Decision Tree Aggregation process for Federated Learning and models
  7. Interpretable Client Decision Tree Aggregation For Federated Learning process (ICDTA4FL process)
  8. (Client's side) Build and send local decision tree
  9. (Server's side) Send the local DTs to the clients
  10. (Client's side) Evaluate the models from other clients
  11. (Server's side) Build the global decision tree
  12. (Client's side) Evaluate the global decision tree
  13. ID3 rules aggregation process and tree building (ICDTA4FL-ID3 model)
  14. Aggregate the rules
  15. Build the global decision tree
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Example of rules generated from the clients' trees when they build an ID3. Client 1 rules are obtained by decomposing the Client's 1 tree into rules, while Client 2 rules are only some rules generated by that client. In the figure appears the condition of the rules, the class predicted by the rule, the number of instances that fit the rule in the client's data, and, in bold, the name given to the rule for a better understanding of the aggregation method.
  • Figure 2: Example of rules generated from the clients' trees when they have built a CART. Client 1 rules are obtained by decomposing the Client's 1 tree into rules, while Client 2 rules are only some rules generated by that client. In the figure appears the condition of the rules, the class predicted by the rule, the number of instances that fit the rule in the client's data, and, in bold, the name given to the rule for a better understanding of the aggregation method.
  • Figure 3: Exploring the impact of adjusting the filter on Accuracy and Macro-F1 scores in a Non-IID distribution on the Nursery Dataset using the ICDTA4FL-ID3 model. The percentile-based filter ignores the trees with metrics that do not surpass such percentile when constructing the global DT.