Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning

Xiaolin Chen, Shuai Zhou, Bei guan, Kai Yang, Hao Fan, Hu Wang, Yongji Wang

TL;DR

This paper protects data privacy and allow the disclosure of feature meaning by concealing decision paths and adapt a communication-efficient secure computation method for inference outputs and improves the interpretability of the model by disclosing the meaning of features while ensuring efficiency and accuracy.

Abstract

The increasing concerns about data privacy and security drive an emerging field of studying privacy-preserving machine learning from isolated data sources, i.e., federated learning. A class of federated learning, vertical federated learning, where different parties hold different features for common users, has a great potential of driving a great variety of business cooperation among enterprises in many fields. In machine learning, decision tree ensembles such as gradient boosting decision trees (GBDT) and random forest are widely applied powerful models with high interpretability and modeling efficiency. However, stateof-art vertical federated learning frameworks adapt anonymous features to avoid possible data breaches, makes the interpretability of the model compromised. To address this issue in the inference process, in this paper, we firstly make a problem analysis about the necessity of disclosure meanings of feature to Guest Party in vertical federated learning. Then we find the prediction result of a tree could be expressed as the intersection of results of sub-models of the tree held by all parties. With this key observation, we protect data privacy and allow the disclosure of feature meaning by concealing decision paths and adapt a communication-efficient secure computation method for inference outputs. The advantages of Fed-EINI will be demonstrated through both theoretical analysis and extensive numerical results. We improve the interpretability of the model by disclosing the meaning of features while ensuring efficiency and accuracy.

Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning

TL;DR

This paper protects data privacy and allow the disclosure of feature meaning by concealing decision paths and adapt a communication-efficient secure computation method for inference outputs and improves the interpretability of the model by disclosing the meaning of features while ensuring efficiency and accuracy.

Abstract

The increasing concerns about data privacy and security drive an emerging field of studying privacy-preserving machine learning from isolated data sources, i.e., federated learning. A class of federated learning, vertical federated learning, where different parties hold different features for common users, has a great potential of driving a great variety of business cooperation among enterprises in many fields. In machine learning, decision tree ensembles such as gradient boosting decision trees (GBDT) and random forest are widely applied powerful models with high interpretability and modeling efficiency. However, stateof-art vertical federated learning frameworks adapt anonymous features to avoid possible data breaches, makes the interpretability of the model compromised. To address this issue in the inference process, in this paper, we firstly make a problem analysis about the necessity of disclosure meanings of feature to Guest Party in vertical federated learning. Then we find the prediction result of a tree could be expressed as the intersection of results of sub-models of the tree held by all parties. With this key observation, we protect data privacy and allow the disclosure of feature meaning by concealing decision paths and adapt a communication-efficient secure computation method for inference outputs. The advantages of Fed-EINI will be demonstrated through both theoretical analysis and extensive numerical results. We improve the interpretability of the model by disclosing the meaning of features while ensuring efficiency and accuracy.

Paper Structure

This paper contains 20 sections, 1 theorem, 10 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Definition
  3. Vertical Federated Decision Tree Ensemble Models
  4. Vertical Federated Inference Problem
  5. Multi-Interactive (MI) Inference
  6. Problem Analysis
  7. Fed-EINI: Proposed Two-Stage Framework
  8. Fed-EINI Algorithm
  9. Stage 1: Parallel Calculation
  10. Stage 2: Synchronization
  11. Extension To Multi-party Scenarios
  12. Analysis of Fed-EINI
  13. Security and Interpretability
  14. Efficiency
  15. Experiment and Results
  16. ...and 5 more sections

Key Result

Proposition 1

For the $Tree_k$, the weight $w'_{(j,k)}$ obtained by taking the intersection of results of the sub-models is unique and equal to the weight $w_{(j,k)}$ obtained by MI inference. That means,

Figures (8)

  • Figure 1: Structure of interactions for the multi-interactive inference framework.
  • Figure 2: An example for MI inference
  • Figure 3: The inference structure of Fed-EINI.
  • Figure 4: An example for Fed-EINI.
  • Figure 5: Multi-party security inference.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Proposition 1
  • proof