Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distilling Deep Reinforcement Learning into Interpretable Fuzzy Rules: An Explainable AI Framework

Sanup S. Araballi, Simon Khan, Chilukuri K. Mohan

Abstract

Deep Reinforcement Learning (DRL) agents achieve remarkable performance in continuous control but remain opaque, hindering deployment in safety-critical domains. Existing explainability methods either provide only local insights (SHAP, LIME) or employ over-simplified surrogates failing to capture continuous dynamics (decision trees). This work proposes a Hierarchical Takagi-Sugeno-Kang (TSK) Fuzzy Classifier System (FCS) distilling neural policies into human-readable IF-THEN rules through K-Means clustering for state partitioning and Ridge Regression for local action inference. Three quantifiable metrics are introduced: Fuzzy Rule Activation Density (FRAD) measuring explanation focus, Fuzzy Set Coverage (FSC) validating vocabulary completeness, and Action Space Granularity (ASG) assessing control mode diversity. Dynamic Time Warping (DTW) validates temporal behavioral fidelity. Empirical evaluation on \textit{Lunar Lander(Continuous)} shows the Triangular membership function variant achieves 81.48\% $\pm$ 0.43\% fidelity, outperforming Decision Trees by 21 percentage points. The framework exhibits statistically superior interpretability (FRAD = 0.814 vs. 0.723 for Gaussian, $p < 0.001$) with low MSE (0.0053) and DTW distance (1.05). Extracted rules such as ``IF lander drifting left at high altitude THEN apply upward thrust with rightward correction'' enable human verification, establishing a pathway toward trustworthy autonomous systems.

Distilling Deep Reinforcement Learning into Interpretable Fuzzy Rules: An Explainable AI Framework

Abstract

Deep Reinforcement Learning (DRL) agents achieve remarkable performance in continuous control but remain opaque, hindering deployment in safety-critical domains. Existing explainability methods either provide only local insights (SHAP, LIME) or employ over-simplified surrogates failing to capture continuous dynamics (decision trees). This work proposes a Hierarchical Takagi-Sugeno-Kang (TSK) Fuzzy Classifier System (FCS) distilling neural policies into human-readable IF-THEN rules through K-Means clustering for state partitioning and Ridge Regression for local action inference. Three quantifiable metrics are introduced: Fuzzy Rule Activation Density (FRAD) measuring explanation focus, Fuzzy Set Coverage (FSC) validating vocabulary completeness, and Action Space Granularity (ASG) assessing control mode diversity. Dynamic Time Warping (DTW) validates temporal behavioral fidelity. Empirical evaluation on \textit{Lunar Lander(Continuous)} shows the Triangular membership function variant achieves 81.48\% 0.43\% fidelity, outperforming Decision Trees by 21 percentage points. The framework exhibits statistically superior interpretability (FRAD = 0.814 vs. 0.723 for Gaussian, ) with low MSE (0.0053) and DTW distance (1.05). Extracted rules such as ``IF lander drifting left at high altitude THEN apply upward thrust with rightward correction'' enable human verification, establishing a pathway toward trustworthy autonomous systems.
Paper Structure (28 sections, 12 equations, 5 figures, 2 tables)

This paper contains 28 sections, 12 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Explainable Reinforcement Learning
  5. Symbolic Policy Distillation
  6. Fuzzy Systems in Reinforcement Learning
  7. Methodology
  8. Problem Formulation
  9. Hierarchical Architecture
  10. Novel Explainability Measures
  11. Experimental Setup
  12. Environment, Teacher, and Data
  13. Baselines and Configurations
  14. Evaluation
  15. Results
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Hierarchical TSK FCS Architecture. Level 1 partitions state space via K-Means clustering into operational regions. Level 2 learns local TSK consequents within each region via Ridge Regression. Final actions obtained through normalized weighted aggregation of active local models.
  • Figure 2: Trajectory alignment: PPO teacher (black solid) vs. FCS surrogate (red dashed) over 50 timesteps. Close tracking demonstrates high behavioral fidelity (DTW = 1.03).
  • Figure 3: Learned Guassian membership functions for Position $X$. K-Means partitioned the dimension into overlapping, locally concentrated regions. Compact support ensures localized activations.
  • Figure 4: Rule activation for sample state. Only Rules 2 and 16 are significantly activated ($\alpha_2 \approx 0.7$, $\alpha_{16} \approx 0.28$), yielding FRAD = 0.89.
  • Figure 5: Fidelity vs. rule count. The 4-rule model achieves peak (97.8%), validating the "Less is More" principle. Beyond 4 rules, over-partitioning degrades performance.