Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean

TL;DR

This work addresses interpretability gaps in deep neural regression by introducing Neural Additive Models for Location, Scale and Shape (NAMLSS), a framework that extends Neural Additive Models (NAMs) to jointly model all distributional parameters of the conditional response. By combining additive subnetwork structure with distributional regression (GAMLSS-like), NAMLSS preserves feature-level interpretability while enabling predictions of uncertainty and distributional shape through parameters such as location, scale, and skewness. The paper presents two architectures for NAMLSS and demonstrates, across synthetic and real-world benchmarks, that NAMLSS often achieves superior predictive performance and offers meaningful distributional insights, including prediction intervals and calibrated uncertainty via log-likelihood and CRPS scores. These results suggest NAMLSS as a practical, interpretable alternative for high-risk applications where understanding the full response distribution is crucial and supports extensions to copulas, mixtures, Bayesian training, and multimodal data streams.

Abstract

Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks, making them the go-to method for problems requiring high-level predictive power. Despite this success, the inner workings of DNNs are often not transparent, making them difficult to interpret or understand. This lack of interpretability has led to increased research on inherently interpretable neural networks in recent years. Models such as Neural Additive Models (NAMs) achieve visual interpretability through the combination of classical statistical methods with DNNs. However, these approaches only concentrate on mean response predictions, leaving out other properties of the response distribution of the underlying data. We propose Neural Additive Models for Location Scale and Shape (NAMLSS), a modelling framework that combines the predictive power of classical deep learning models with the inherent advantages of distributional regression while maintaining the interpretability of additive models. The code is available at the following link: https://github.com/AnFreTh/NAMpy

Figures (8)

• Figure 1: Johnson's S$_U$ Distribution: Simulated Johnson's S$_U$ distribution and the fit of a simple NAMLSS (see Figure \ref{['fig:namlss_structure1']}) and a MLP. While the MLP achieves an impressive fit concerning the quadratic loss, it clearly cannot capture the underlying distribution adequately.
• Figure 2: The network structure of a NAMLSS model. Each input variable is handled by a different neural network with $k$ outputs for each distributional parameter. $h^{(k)}$ are different activation functions depending on the distributional parameter that is modelled. E.g. a quadratic transformation for modelling the variance in a normally distributed variable to ensure the non-negativity constraint. The presented structure demonstrates a NAMLSS modelling a distribution with two parameters, e.g. a normal distribution.
• Figure 3: NAMLSS distributional parameter prediction for a Johnson's S$_U$ Distribution over 150 runs. NAMLSS accurately detects the feature effects of $x_1$ and $x_2$ on all distributional parameters, $\theta^{(1)}$ the mean, $\theta^{(2)}$ the standard deviation, $\theta^{(3)}$ the skewness and $\theta^{(4)}$ the tailweight.
• Figure 4: California Housing: Graphs for median income and population respectively learned by the NAMLSS model. We see an increase in housing prices with a larger median income. Additionally, we find a larger variance in housing prices in less densely populated areas.
• Figure 5: California Housing: Graphs for longitude and latitude respectively learned by the NAMLSS model. NAMLSS captures changes in mean as well as variance. Therefore, the plotted standard deviations change in dependence of the longitude and latitude. The house price jumps around the location of Los Angeles are depictable. Additionally, we find a decrease in variance for areas further away from the large cities.