Higher-order Neural Additive Models: An Interpretable Machine Learning Model with Feature Interactions
Minkyu Kim, Hyun-Soo Choi, Jinho Kim
TL;DR
HONAM extends Neural Additive Models to capture higher-order feature interactions while maintaining interpretability. The approach stacks per-feature representations and introduces a recursive, efficient high-order interaction module with complexity $O(k m t)$ to avoid powered terms, enabling interactions up to order $t$ without exponential feature explosion. Empirical results across 10 public datasets show HONAM often outperforms NAM, NodeGAM, and EBM and approaches black-box models in predictive performance, with rich local and global interpretations demonstrated on datasets like FICO. The work highlights HONAM’s potential for high-stakes domains where strong explanations of complex interactions are crucial, and provides a publicly available implementation for broader adoption.
Abstract
Neural Additive Models (NAMs) have recently demonstrated promising predictive performance while maintaining interpretability. However, their capacity is limited to capturing only first-order feature interactions, which restricts their effectiveness on real-world datasets. To address this limitation, we propose Higher-order Neural Additive Models (HONAMs), an interpretable machine learning model that effectively and efficiently captures feature interactions of arbitrary orders. HONAMs improve predictive accuracy without compromising interpretability, an essential requirement in high-stakes applications. This advantage of HONAM can help analyze and extract high-order interactions present in datasets. The source code for HONAM is publicly available at https://github.com/gim4855744/HONAM/.