Multiplicative-Additive Constrained Models:Toward Joint Visualization of Interactive and Independent Effects
Fumin Wang
TL;DR
This work tackles interpretability in high stakes settings by addressing GAMs limited to independent effects and CESR's entangled multiplicative terms. It introduces Multiplicative-Additive Constrained Models (MACMs), a framework that combines a multiplicative component with an additive component to decouple coefficients and expand hypothesis space, using visualizable univariate shape functions. Neural network based MACMs (MACMs(NNs)) outperform CESR and state of the art NN based GAMs across regression and classification tasks, while also preserving interpretability through shape function visualizations and dynamic influence curves. Overall, MACMs provide a practical bridge between multiplicative interaction modeling and additive interpretable structure, delivering improved predictive performance without sacrificing transparent decision reasoning in domains such as healthcare.
Abstract
Interpretability is one of the considerations when applying machine learning to high-stakes fields such as healthcare that involve matters of life safety. Generalized Additive Models (GAMs) enhance interpretability by visualizing shape functions. Nevertheless, to preserve interpretability, GAMs omit higher-order interaction effects (beyond pairwise interactions), which imposes significant constraints on their predictive performance. We observe that Curve Ergodic Set Regression (CESR), a multiplicative model, naturally enables the visualization of its shape functions and simultaneously incorporates both interactions among all features and individual feature effects. Nevertheless, CESR fails to demonstrate superior performance compared to GAMs. We introduce Multiplicative-Additive Constrained Models (MACMs), which augment CESR with an additive part to disentangle the intertwined coefficients of its interactive and independent terms, thus effectively broadening the hypothesis space. The model is composed of a multiplicative part and an additive part, whose shape functions can both be naturally visualized, thereby assisting users in interpreting how features participate in the decision-making process. Consequently, MACMs constitute an improvement over both CESR and GAMs. The experimental results indicate that neural network-based MACMs significantly outperform both CESR and the current state-of-the-art GAMs in terms of predictive performance.