Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Logistic Models Really Interpretable?

Danial Dervovic, Freddy Lécué, Nicolás Marchesotti, Daniele Magazzeni

TL;DR

This paper questions the interpretability of logistic regression in high-stakes settings and demonstrates that human users struggle to simulate LR behavior from coefficients. It introduces Linearised Additive Models (LAMs), a universal 3-piece, clipped linear approximation to the sigmoid that converts trained additive models into probability-space, interpretable forms with provable optimality. Empirically, LAMs incur only small ROC-AUC and calibration penalties compared with logistic models across public credit datasets, while the user study shows significantly better interpretability for LAMs in human-simulation tasks. The work highlights a practical trade-off: modest performance loss in exchange for markedly improved interpretability, and suggests cautions and directions for future research on model explanations and broader GAM applicability.

Abstract

The demand for open and trustworthy AI models points towards widespread publishing of model weights. Consumers of these model weights must be able to act accordingly with the information provided. That said, one of the simplest AI classification models, Logistic Regression (LR), has an unwieldy interpretation of its model weights, with greater difficulties when extending LR to generalised additive models. In this work, we show via a User Study that skilled participants are unable to reliably reproduce the action of small LR models given the trained parameters. As an antidote to this, we define Linearised Additive Models (LAMs), an optimal piecewise linear approximation that augments any trained additive model equipped with a sigmoid link function, requiring no retraining. We argue that LAMs are more interpretable than logistic models -- survey participants are shown to solve model reasoning tasks with LAMs much more accurately than with LR given the same information. Furthermore, we show that LAMs do not suffer from large performance penalties in terms of ROC-AUC and calibration with respect to their logistic counterparts on a broad suite of public financial modelling data.

Are Logistic Models Really Interpretable?

TL;DR

This paper questions the interpretability of logistic regression in high-stakes settings and demonstrates that human users struggle to simulate LR behavior from coefficients. It introduces Linearised Additive Models (LAMs), a universal 3-piece, clipped linear approximation to the sigmoid that converts trained additive models into probability-space, interpretable forms with provable optimality. Empirically, LAMs incur only small ROC-AUC and calibration penalties compared with logistic models across public credit datasets, while the user study shows significantly better interpretability for LAMs in human-simulation tasks. The work highlights a practical trade-off: modest performance loss in exchange for markedly improved interpretability, and suggests cautions and directions for future research on model explanations and broader GAM applicability.

Abstract

The demand for open and trustworthy AI models points towards widespread publishing of model weights. Consumers of these model weights must be able to act accordingly with the information provided. That said, one of the simplest AI classification models, Logistic Regression (LR), has an unwieldy interpretation of its model weights, with greater difficulties when extending LR to generalised additive models. In this work, we show via a User Study that skilled participants are unable to reliably reproduce the action of small LR models given the trained parameters. As an antidote to this, we define Linearised Additive Models (LAMs), an optimal piecewise linear approximation that augments any trained additive model equipped with a sigmoid link function, requiring no retraining. We argue that LAMs are more interpretable than logistic models -- survey participants are shown to solve model reasoning tasks with LAMs much more accurately than with LR given the same information. Furthermore, we show that LAMs do not suffer from large performance penalties in terms of ROC-AUC and calibration with respect to their logistic counterparts on a broad suite of public financial modelling data.
Paper Structure (48 sections, 3 theorems, 20 equations, 5 figures, 8 tables)

This paper contains 48 sections, 3 theorems, 20 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work.
  4. Structure.
  5. Notation.
  6. Logistic and Linear Probability Modelling
  7. Logistic Modelling and Interpretability
  8. Motivating Example.
  9. Linearised Additive Models (LAMs)
  10. Motivating Example Revisited.
  11. Performance Comparison
  12. Experimental Setup
  13. Models.
  14. Datasets.
  15. Performance Metrics
  16. ...and 33 more sections

Key Result

Theorem 2.4

Let $\mathrm{PL}_3$ be the space of 3-piece piecewise linear functions of one variable and $\mathcal{X} = \mathbb{R}^d$. For any LR model $\hat{y}(\vb*{x}) = \sigma(f(\vb*{x})) = \sigma(\beta_0 + \sum_{i = 1}^d \beta_i x_i)$ on $\mathcal{X}$, an approximator $\widetilde{\sigma}(f(\vb*{x}))$ is defin with $\widetilde{\sigma}(z; \alpha^\star) := \Pi_{[0, 1]}(\frac{1}{2}(1 + \frac{z}{\alpha^\star}))$

Figures (5)

  • Figure 1: The cost of misinterpration of model coefficients as explanations. Alice and Bob receive the same explanation, but incur a different change in model output, ultimately leading to different outcomes.
  • Figure 2: Optimal approximation $\widetilde{\sigma}(x; \alpha \approx 2.5996)$ to sigmoid function $\sigma(x)$. The parameter $\alpha$ corresponds the half the width along the $x$-axis of the middle line segment in the piecewise linear function.
  • Figure 3: Critical Difference diagrams for AUC, ECE and MCE. The $x$-axis represents the mean rank averaged over all datasets, with each classifier's mean rank reported adjacent to its name (lower rank $\equiv$ better). Classifiers connected by an edge cannot be distinguished with significance $\alpha=0.05$.
  • Figure 4: Summary of responses to User Survey comparing interpretability of LR models against LAM-LR. Users predict the change in direction of model output correctly at a slightly higher rate for linearised models (left). When users are asked about the magnitude of this change, users fare overwhelmingly better using LAM-LR as opposed to LR (center). When asked about which model they found easiest to use (right), the majority of users said neither model, with slightly fewer opting for LAM-LR and only one for LR.
  • Figure 5: Squared error for linear approximation to sigmoid (left panel) and optimal (in square error) clipped linear approximation $\widetilde{\sigma}(x; \alpha \approx 2.5996)$ to sigmoid function $\sigma(x)$ (right panel).

Theorems & Definitions (12)

  • Definition 2.1: Logistic Additive Model
  • Definition 2.2: Linearised Additive Models (LAM)
  • Remark 2.3
  • Theorem 2.4: LAM Optimality
  • proof : Proof (Sketch)
  • Definition A.1: Piecewise Linear Functions
  • Definition A.2: $\mathrm{PL}_n$
  • Proposition A.3
  • proof
  • Lemma A.4
  • ...and 2 more