Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partition of Unity Neural Networks for Interpretable Classification with Explicit Class Regions

Akram Aldroubi

TL;DR

Partition of Unity Neural Networks (PUNN) replace softmax with a partition-of-unity architecture, yielding class probabilities directly from learned region functions $h_i(x)$ that satisfy $\sum_i h_i(x)=1$. The authors prove density of PUNN in the space of continuous probability maps and show flexible gate parameterizations, including shape-informed priors, enabling parameter-efficient, interpretable models. Empirically, PUNN matches or nearly matches standard MLP accuracy on UCI benchmarks and MNIST, with substantial parameter reductions when geometric priors are available. The approach provides built-in interpretability through explicit accept/reject gate structure and partition functions, offering a transparent alternative to post-hoc explanations with competitive practical impact for high-stakes classification tasks.

Abstract

Despite their empirical success, neural network classifiers remain difficult to interpret. In softmax-based models, class regions are defined implicitly as solutions to systems of inequalities among logits, making them difficult to extract and visualize. We introduce Partition of Unity Neural Networks (PUNN), an architecture in which class probabilities arise directly from a learned partition of unity, without requiring a softmax layer. PUNN constructs $k$ nonnegative functions $h_1, \ldots, h_k$ satisfying $\sum_i h_i(x) = 1$, where each $h_i(x)$ directly represents $P(\text{class } i \mid x)$. Unlike softmax, where class regions are defined implicitly through coupled inequalities among logits, each PUNN partition function $h_i$ directly defines the probability of class $i$ as a standalone function of $x$. We prove that PUNN is dense in the space of continuous probability maps on compact domains. The gate functions $g_i$ that define the partition can use various activation functions (sigmoid, Gaussian, bump) and parameterizations ranging from flexible MLPs to parameter-efficient shape-informed designs (spherical shells, ellipsoids, spherical harmonics). Experiments on synthetic data, UCI benchmarks, and MNIST show that PUNN with MLP-based gates achieves accuracy within 0.3--0.6\% of standard multilayer perceptrons. When geometric priors match the data structure, shape-informed gates achieve comparable accuracy with up to 300$\times$ fewer parameters. These results demonstrate that interpretable-by-design architectures can be competitive with black-box models while providing transparent class probability assignments.

Partition of Unity Neural Networks for Interpretable Classification with Explicit Class Regions

TL;DR

Partition of Unity Neural Networks (PUNN) replace softmax with a partition-of-unity architecture, yielding class probabilities directly from learned region functions that satisfy . The authors prove density of PUNN in the space of continuous probability maps and show flexible gate parameterizations, including shape-informed priors, enabling parameter-efficient, interpretable models. Empirically, PUNN matches or nearly matches standard MLP accuracy on UCI benchmarks and MNIST, with substantial parameter reductions when geometric priors are available. The approach provides built-in interpretability through explicit accept/reject gate structure and partition functions, offering a transparent alternative to post-hoc explanations with competitive practical impact for high-stakes classification tasks.

Abstract

Despite their empirical success, neural network classifiers remain difficult to interpret. In softmax-based models, class regions are defined implicitly as solutions to systems of inequalities among logits, making them difficult to extract and visualize. We introduce Partition of Unity Neural Networks (PUNN), an architecture in which class probabilities arise directly from a learned partition of unity, without requiring a softmax layer. PUNN constructs nonnegative functions satisfying , where each directly represents . Unlike softmax, where class regions are defined implicitly through coupled inequalities among logits, each PUNN partition function directly defines the probability of class as a standalone function of . We prove that PUNN is dense in the space of continuous probability maps on compact domains. The gate functions that define the partition can use various activation functions (sigmoid, Gaussian, bump) and parameterizations ranging from flexible MLPs to parameter-efficient shape-informed designs (spherical shells, ellipsoids, spherical harmonics). Experiments on synthetic data, UCI benchmarks, and MNIST show that PUNN with MLP-based gates achieves accuracy within 0.3--0.6\% of standard multilayer perceptrons. When geometric priors match the data structure, shape-informed gates achieve comparable accuracy with up to 300 fewer parameters. These results demonstrate that interpretable-by-design architectures can be competitive with black-box models while providing transparent class probability assignments.
Paper Structure (59 sections, 2 theorems, 39 equations, 4 figures, 10 tables)

This paper contains 59 sections, 2 theorems, 39 equations, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Partition of Unity Methods
  4. Interpretable Machine Learning
  5. Approximation of Probability Maps
  6. Partition of Unity Neural Networks
  7. Architecture Definition
  8. PUNN for Classification
  9. Hierarchical Decision Structure
  10. Theoretical Analysis
  11. Density of PUNN in the space of continuous probability maps
  12. Flexibility in Gate Function Parameterization
  13. Experiments
  14. Experimental Setup
  15. Synthetic Datasets
  16. ...and 44 more sections

Key Result

Proposition 1

Let $k \ge 2$. For every $x \in \mathbb{R}^d$, the partition functions satisfy

Figures (4)

  • Figure 1: Decision boundaries learned by PUNN on synthetic datasets. Rows: Moons, Circles, XOR, Helix. Columns: Sigma, Bump, Gaussian gates. Background color indicates $h_1(x)$ (blue = 1, red = 0); black contour shows the decision boundary at $h_1(x) = 0.5$.
  • Figure 2: Partition functions $h_0(x)$ and $h_1(x)$ learned by PUNN-Sigma. Left column: $h_0(x)$; right column: $h_1(x)$. Rows correspond to Moons, Circles, XOR, and Helix datasets. The complementary structure ($h_0 + h_1 = 1$) is evident, enabling direct probabilistic interpretation.
  • Figure 3: Decision boundaries on Circles dataset. Left: Shape-informed spherical shell (4 parameters) learns a smooth circular boundary. Right: MLP (1,218 parameters) learns an irregular boundary. The shape-informed gate achieves 304$\times$ parameter reduction while producing a more interpretable decision region.
  • Figure 4: Partition functions learned by a spherical shell gate on Circles. Left: $h_0(x)$ assigns high probability to the inner circle. Middle: $h_1(x) = 1 - h_0(x)$ assigns high probability to the outer ring. Right: Decision boundary at $h_0(x) = 0.5$. The complementary structure ($h_0 + h_1 = 1$) enables direct probabilistic interpretation with only 4 parameters.

Theorems & Definitions (9)

  • Proposition 1: Partition of Unity Property
  • proof
  • Remark 2: Extension to Multi-Modal Classes
  • Definition 3: $(k-1)$-Simplex
  • Definition 4: Probability Map
  • Theorem 5
  • proof
  • Remark 6: Partition of Unity Preservation
  • Definition 7: Direction-Dependent Shell Gate