S-SIRUS: an explainability algorithm for spatial regression Random Forest

TL;DR

This paper introduces S-SIRUS, a spatially aware explainability algorithm that extracts a compact, stable set of rules from RF-GLS for regression with spatially dependent data. By discretizing predictors, limiting tree depth, and applying non-negative ridge aggregation, S-SIRUS produces interpretable rules and a robust large-scale component, with cross-validated tuning of the sparsity parameter $p_0$ to balance stability and predictive power. In simulations based on a pseudo-real AgrImOnIA dataset, S-SIRUS outperforms its non-spatial counterpart SIRUS in settings with stronger spatial dependence, yielding shorter rule lists and comparable or better predictive performance, especially when coupled with residual kriging for final y predictions. The work demonstrates a practical pathway to explain RF in spatial contexts, with implications for environmental science and geostatistical modeling, and provides code and tools to reproduce the approach.

Abstract

Random Forest (RF) is a widely used machine learning algorithm known for its flexibility, user-friendliness, and high predictive performance across various domains. However, it is non-interpretable. This can limit its usefulness in applied sciences, where understanding the relationships between predictors and response variable is crucial from a decision-making perspective. In the literature, several methods have been proposed to explain RF, but none of them addresses the challenge of explaining RF in the context of spatially dependent data. Therefore, this work aims to explain regression RF in the case of spatially dependent data by extracting a compact and simple list of rules. In this respect, we propose S-SIRUS, a spatial extension of SIRUS, the latter being a well-established regression rule algorithm able to extract a stable and short list of rules from the classical regression RF algorithm. A simulation study was conducted to evaluate the explainability capability of the proposed S-SIRUS, in comparison to SIRUS, by considering different levels of spatial dependence among the data. The results suggest that S-SIRUS exhibits a higher test predictive accuracy than SIRUS when spatial correlation is present. Moreover, for higher levels of spatial correlation, S-SIRUS produces a shorter list of rules, easing the explanation of the mechanism behind the predictions.

Figures (5)

• Figure 1: Spatial sites coloured according to the values of the simulated response variable ($\log(\text{PM}_{10})$) for Scenario A; 400 training observations (circle) and 100 test observations (triangle).
• Figure 2: Unexplained variance (top panel) and stability (bottom panel) versus the number of rules for SIRUS (left panels) and S-SIRUS (right panels) in Scenario A for a fine grid of $p_0$, assessed via standard CV with $K=10$ folds. The results are averaged and bars show the variability of the metrics across 10 repetitions. The optimal $p_0$ value is the median $p_0$ value across the 10 CV repetitions (0.0257 for SIRUS and 0.0296 for S-SIRUS).
• Figure 3: Estimated exponential variograms $\hat{\gamma} (h)$ for SIRUS (black solid line) and S-SIRUS (blue dashed line) residuals, with associated bootstrap-based confidence regions (shadow area) for the three scenarios.
• Figure 4: Unexplained variance (top panel) and stability (bottom panel) versus the number of rules for SIRUS (left panels) and S-SIRUS (right panels) in Scenario B (SNR = 1) for a fine grid of $p_0$, assessed via standard CV with $K=10$ folds. The results are averaged and bars show the variability of the metrics across 10 repetitions. The optimal $p_0$ value is the median $p_0$ value across the 10 CV repetitions (0.0257 for SIRUS and 0.0232 for S-SIRUS).
• Figure 5: Unexplained variance (top panel) and stability (bottom panel) versus the number of rules for SIRUS (left panels) and S-SIRUS (right panels) in Scenario C (SNR = 2) for a fine grid of $p_0$, assessed via standard CV with $K=10$ folds. The results are averaged and bars show the variability of the metrics across 10 repetitions. The optimal $p_0$ value is the median $p_0$ value across the 10 CV repetitions (0.0616 for SIRUS and 0.0196 for S-SIRUS).