SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries

TL;DR

SplineCam tackles the challenge of exactly visualizing and characterizing the geometry of deep networks with CPWL activations by providing a provably exact, sampling-free method to enumerate input-space partitions and the network's decision boundary. Grounded in a MASO/CPA formulation, it expresses layer mappings as region-wise affine transforms with unique per-region derivatives $\boldsymbol{q}^\ell_\omega$, and derives an algorithm that builds a hyperplane intersection graph to enumerate regions and back-project layer hyperplanes, yielding the boundary $\cup_{\omega\in\Omega} \{ \text{proj}_{\omega}(h_1^L) \cap \omega \}$. The approach enables visualization, partition statistics, and boundary sampling for architecture comparison, generalization analysis, and INR/SDF studies. Empirically, it reveals how architecture (e.g., CNN vs MLP), data augmentation, and positional encoding affect partition density and decision boundaries, with applications to implicit neural representations and signed distance functions, offering a tool for boundary-aware initialization and diagnostics.

Abstract

Current Deep Network (DN) visualization and interpretability methods rely heavily on data space visualizations such as scoring which dimensions of the data are responsible for their associated prediction or generating new data features or samples that best match a given DN unit or representation. In this paper, we go one step further by developing the first provably exact method for computing the geometry of a DN's mapping - including its decision boundary - over a specified region of the data space. By leveraging the theory of Continuous Piece-Wise Linear (CPWL) spline DNs, SplineCam exactly computes a DNs geometry without resorting to approximations such as sampling or architecture simplification. SplineCam applies to any DN architecture based on CPWL nonlinearities, including (leaky-)ReLU, absolute value, maxout, and max-pooling and can also be applied to regression DNs such as implicit neural representations. Beyond decision boundary visualization and characterization, SplineCam enables one to compare architectures, measure generalizability and sample from the decision boundary on or off the manifold. Project Website: bit.ly/splinecam.

Figures (20)

• Figure 1: Exact visualization of the decision boundary and partition geometry of a 3D neural signed distance field (SDF). (Top left) Surface normals obtained from the learned signed distance field with annotations indicating slices used for visualization. For each of the slices, we can see the spline partition geometry of the learned SDF- each contiguous line represents a neuron, on either side of which it gets activated/deactivated. Neurons from different depths of the network create a partitioning of the input space into 'linear regions'. Here the colored lines represent the decision boundary learned by the SDF. Note that while the final neuron obtains the decision boundary, many neurons place their boundaries close to the ground truth surface to obtain the final SDF representation.
• Figure 2: Visual depiction of Eq. \ref{['eq:CPA']} with a toy affine spline mapping $S : \mathbb{R}^2 \rightarrow \mathbb{R}^3$. Left. Input space partition $\Omega$ made of multiple convex regions shown with different colors and with boundaries shown in black. Right. Affine spline image $Im(S)$ which is a continuous piecewise affine surface composed of the input space regions affinely transformed by the per-region affine mappings. Colors maintain correspondence from the left to the right.
• Figure 3: Given an input domain $P$ and a set of hyperplanes $\mathcal{H}$, SplineCam first produces a graph $G$ using all the edge and hyperplane intersections (as in Alg. \ref{['alg:findPartitions']}). To find all the cycles in the graph, we select a boundary edge $e_s$ (blue arrow), do a breadth-first search (BFS) to find the shortest path through the graph between vertices of $e_s$ and obtain the corresponding cycle (blue). The edges obtained via BFS are enqueued and the search is repeated for each. Each non-boundary edge is allowed to be traversed twice, once from either direction (see Alg. \ref{['alg:findCycles']}). Once new regions are found, we back-project deeper layer hyperplanes, compute partition graphs, and repeat.
• Figure 4: SplineCam visualizations of neurons from different layers of an MLP trained with (top) and without (bottom) periodic position encoding on a 2D image fitting task. All the neurons are visualized in the input space, color coded by the same color, and one neuron from each layer is highlighted in red. The trained MLP has a width of $10$ and depth of $5$ and has ReLU activations for every layer. For the positionally encoded (PE) network, boundaries of some neurons seem to be periodically repeating in the input domain, significantly increasing the number of unique $\omega$ where the ReLU is active. The increased weight sharing, i.e., same weights/neurons being used to represent/fit non-contiguous parts of the learned function, could be a possible reason for improved convergence of PE MLP nowlan1992simplifying.
• Figure 5: Visualization of the decision boundary and partition geometry of a 2D neural SDF with width $256$ and depth $6$. A single training image is thresholded at $0.01$ and $0.5$ to create two signed distance fields (top-middle and top-right), on which an MLP is trained. We use SplineCam to obtain the analytical zero level set (decision boundary, in red) and also visualize the partition geometry (bottom). Note that even with identical architecture, the partition density differs significantly based on the task complexity.