Representing Piecewise-Linear Functions by Functions with Minimal Arity

TL;DR

This work investigates the minimal arity required to represent a continuous piecewise-linear function $F:\mathbb{R}^{n}\to \mathbb{R}$ as a sum of maxima of affine functions. It introduces a polyhedral-geometry framework based on piecewise-constant functions, flags, and delta functions, establishing that $F$ admits a representation with at most $k$ arguments if and only if the delta function $\Delta_{\nabla F}(\mathcal{H})$ is constant for all flags of length $k-1$, and shows how to compute the minimal $k^{*}$ via a finite-flag analysis. A constructive decomposition is given that expresses $F$ as a sum of components each using at most $k+1$ arguments, linking these results to ReLU network depth bounds and revealing that matching tessellations do not guarantee low-arity representations. The work further clarifies that the interplay between the input-space tessellation and function gradient structure governs the efficiency of piecewise-linear representations.

Abstract

Any continuous piecewise-linear function $F\colon \mathbb{R}^{n}\to \mathbb{R}$ can be represented as a linear combination of $\max$ functions of at most $n+1$ affine-linear functions. In our previous paper [``Representing piecewise linear functions by functions with small arity'', AAECC, 2023], we showed that this upper bound of $n+1$ arguments is tight. In the present paper, we extend this result by establishing a correspondence between the function $F$ and the minimal number of arguments that are needed in any such decomposition. We show that the tessellation of the input space $\mathbb{R}^{n}$ induced by the function $F$ has a direct connection to the number of arguments in the $\max$ functions.

Figures (2)

• Figure 1: Two CPWL functions, $G_{1}(x, y) = \max(0, x, y)$ (first row), and $G_{2}(x, y) = \max(0, -x, -y)$ (second row). Each row shows the function $\mathbb R^{2}\to \mathbb R$ as a three-dimensional plot (first column) and the function's tessellation as a contour plot (second column). For a hyperplane $H_{1} = \{(x, y) \in \mathbb R^{2}\mid x =y \}$, the delta functions $\Delta_{\nabla G_1}((H_1))$ and $\Delta_{\nabla G_2}((H_1))$ are non-constant.
• Figure 2: Two CPWL functions, $G_{3}(x, y) = \max(0, x, y) + \max(0, -x, -y)$ (first row), and $G_{4}(x, y) = 6\max(0, x, y) + \max(0, -x, -y)$ (second row). Each row shows the function $\mathbb R^{2}\to \mathbb R$ as a three-dimensional plot (first column) and the function's tessellation as a contour plot (second column). For any flag $\mathcal{H}$ of length 1, the delta function $\Delta_{\nabla G_3}(\mathcal{H})$ is constant. For a hyperplane $H_{1}~=~\{(x, y) \in \mathbb R^{2}\mid x =y \}$, the delta function $\Delta_{\nabla G_4}((H_1))$ is non-constant.

Theorems & Definitions (34)

• Example 1.1
• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Definition 2.5
• Lemma 2.6
• proof
• Lemma 2.7
• proof