Post-hoc Interpretability Illumination for Scientific Interaction Discovery

TL;DR

iKF introduces a post-hoc, model-agnostic interpretability framework that fixes a variable as a King at the root of King's Forests and uses depth-bounded trees with PVIM-based ranking to uncover multi-order interactions. The approach defines a King-centric workflow, two inference metrics (King's PVIM and Path Reproduction Count), and a three-type taxonomy (Accompanied, Synergistic, Hierarchical) to classify interactions, iterating to identify successive Kings until a stopping criterion is met. Evaluations on synthetic interaction tasks and a Drosophila TF dataset show that iKF more reliably recovers high-order interactions than DC-SIS and iRF, and can rediscover known biological interaction mechanisms, demonstrating practical utility for explainable modeling and scientific discovery. The work suggests broad applicability to complex systems where interpretable interaction structures drive understanding and discovery across disciplines.

Abstract

Model interpretability and explainability have garnered substantial attention in recent years, particularly in decision-making applications. However, existing interpretability tools often fall short in delivering satisfactory performance due to limited capabilities or efficiency issues. To address these challenges, we propose a novel post-hoc method: Iterative Kings' Forests (iKF), designed to uncover complex multi-order interactions among variables. iKF iteratively selects the next most important variable, the "King", and constructs King's Forests by placing it at the root node of each tree to identify variables that interact with the "King". It then generates ranked short lists of important variables and interactions of varying orders. Additionally, iKF provides inference metrics to analyze the patterns of the selected interactions and classify them into one of three interaction types: Accompanied Interaction, Synergistic Interaction, and Hierarchical Interaction. Extensive experiments demonstrate the strong interpretive power of our proposed iKF, highlighting its great potential for explainable modeling and scientific discovery across diverse scientific fields.

Figures (2)

• Figure 1: iKF Overview: Iteratively select new Kings until $|S_{i}|$ is sufficiently small. See definition of $S_{i}$ and $I_{i}$ in step (4) below. (a) Use a series of disposable King's Forests to update the variables' weights ${\bm{w}}^{(i)}$, and rank ${\bm{w}}^{(i)}$ to select the Core Team for the $i$-th King. (b) Build $D$ King's Forests to uncover $d$-order interactions by analyzing depth-$d$ tree paths for the $i$-th King.
• Figure 2: Minimum recovery size (MRS) across various quantiles.