Müntz-Szász Networks: Neural Architectures with Learnable Power-Law Bases

TL;DR

This work introduces Müntz-Szász Networks (MSN), a neural architecture that replaces fixed smooth activations with learnable power-law bases to efficiently represent singular and fractional-power functions such as $|x|^α$. Grounded in the Müntz-Szász theorem, MSN learns exponents $μ_k$ and $λ_k$ alongside coefficients to achieve universal approximation with favorable rates, notably yielding $\mathcal{O}(δ^2)$ error when the learned exponent is within $δ$ of the target power and outperforming standard MLPs by large margins on singular targets. The framework includes a bounded exponent parameterization, a Müntz divergence regularizer, and stabilization tricks for training, yielding 3–6× improvements on PINN benchmarks and 5–8× reductions in supervised regression error with 5–10× fewer parameters for singular function classes. Importantly, the learned exponents provide interpretable insights into the underlying solution structure, enabling physics-informed modeling that aligns with known singularity characters. This theory-guided architectural design demonstrates a practical path to dramatically improve performance on scientifically-m motivated function classes and opens avenues for extensions to higher dimensions and fractional PDEs.

Abstract

Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce Müntz-Szász Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes $φ(x) = \sum_k a_k |x|^{μ_k} + \sum_k b_k \mathrm{sign}(x)|x|^{λ_k}$, where the exponents $\{μ_k, λ_k\}$ are learned alongside the coefficients. We prove that MSN inherits universal approximation from the Müntz-Szász theorem and establish novel approximation rates: for functions of the form $|x|^α$, MSN achieves error $\mathcal{O}(|μ- α|^2)$ with a single learned exponent, whereas standard MLPs require $\mathcal{O}(ε^{-1/α})$ neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.

Figures (13)

• Figure 1: Müntz-Szász Network architecture. (a) A Müntz edge computes $\phi(x) = \sum_k a_k |x|^{\mu_k} + \sum_k b_k \operatorname{sign}(x)|x|^{\lambda_k}$ with learnable exponents $\bm{\mu}, \bm{\lambda}$ and coefficients $\bm{a}, \bm{b}$. (b) An MSN layer connects all input-output pairs via Müntz edges with shared exponents. (c) Full MSN stacks layers with inter-layer nonlinearities ($\tanh$). Unlike standard networks with fixed activations, MSN learns the functional form of each edge.
• Figure 2: Comparison of MLP and MSN architectures. MLP uses fixed activation functions (ReLU, tanh) at nodes and learns only weights on edges. MSN uses learnable power functions $|x|^{\mu_k}$ on edges, learning both coefficients and exponents. This enables efficient representation of singular functions.
• Figure 3: Error landscape $E(\mu, \alpha) = \mathcal{O}((\mu - \alpha)^2)$. (Left) 3D surface showing the $L^2$ approximation error as a function of learned exponent $\mu$ and true exponent $\alpha$. The red line marks the zero-error diagonal where $\mu = \alpha$. (Right) Contour view with a gradient descent trajectory showing how MSN learns $\mu \to \alpha$. When the learned exponent matches the true exponent, the error is exactly zero, this is the key insight enabling MSN's dramatic efficiency gains over MLPs.
• Figure 4: Supervised regression analysis. (a) MSN achieves the best accuracy-efficiency trade-off for singular functions, with clear task separation showing when each architecture excels. (b) Learned exponents are interpretable and match the target function structure.
• Figure 5: PINN: singular ODE $u' = 1/(2\sqrt{x})$. (a) MSN closely matches the true $\sqrt{x}$ solution. (b) Error is lowest near $x = 0$ where the singularity occurs. (c) Learned exponents cluster near 0.5, directly reflecting the solution structure.

Theorems & Definitions (24)

• Theorem 1: Müntz-Szász, 1914
• Definition 1: Müntz System
• Theorem 2: Full Müntz Theorem
• Remark 1: Relationship to Classical Theory
• Definition 2: Müntz Edge
• Definition 3: MSN Layer
• Definition 4: Müntz-Szász Network
• Definition 5: Bounded Exponent Map
• Proposition 1: Properties of Bounded Parameterization
• Definition 6: Müntz Divergence