Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Rectifiable Measures through Neural Networks

Erwin Riegler, Alex Bühler, Yang Pan, Helmut Bölcskei

TL;DR

This work proves that measures supported on low-dimensional, rectifiable sets can be universally approximated by push-forwards of the one-dimensional Lebesgue measure through ReLU neural networks with quantized, bounded weights. The key insight is that the intrinsic, not ambient, dimension $m$ governs the approximation rate, with a network count bounded by $2^{b(\varepsilon)}$ where $b(\varepsilon)=\mathcal{O}(\varepsilon^{-m}\log^2(\varepsilon))$, and this extends to countably $m$-rectifiable measures under an exponential-decay assumption on the components. The authors provide constructive space-filling schemes that couple a Lipschitz map approximant $f$ with a space-filling network $\Sigma$ to produce a push-forward $\Psi\#\mathcal{L}^{(1)}|_{[0,1]}$ closely approximating the target measure in Wasserstein distance. This yields universality results with explicit Lipschitz and architectural bounds, improves previous rates tied to ambient dimension, and demonstrates space-filling neural constructions for generating measures on low-dimensional geometric objects. The results have potential implications for synthetic data generation, geometric measure approximation, and learning measures supported on manifolds or unions of Lipschitz images.

Abstract

We derive universal approximation results for the class of (countably) $m$-rectifiable measures. Specifically, we prove that $m$-rectifiable measures can be approximated as push-forwards of the one-dimensional Lebesgue measure on $[0,1]$ using ReLU neural networks with arbitrarily small approximation error in terms of Wasserstein distance. What is more, the weights in the networks under consideration are quantized and bounded and the number of ReLU neural networks required to achieve an approximation error of $\varepsilon$ is no larger than $2^{b(\varepsilon)}$ with $b(\varepsilon)=\mathcal{O}(\varepsilon^{-m}\log^2(\varepsilon))$. This result improves Lemma IX.4 in Perekrestenko et al. as it shows that the rate at which $b(\varepsilon)$ tends to infinity as $\varepsilon$ tends to zero equals the rectifiability parameter $m$, which can be much smaller than the ambient dimension. We extend this result to countably $m$-rectifiable measures and show that this rate still equals the rectifiability parameter $m$ provided that, among other technical assumptions, the measure decays exponentially on the individual components of the countably $m$-rectifiable support set.

Generating Rectifiable Measures through Neural Networks

TL;DR

This work proves that measures supported on low-dimensional, rectifiable sets can be universally approximated by push-forwards of the one-dimensional Lebesgue measure through ReLU neural networks with quantized, bounded weights. The key insight is that the intrinsic, not ambient, dimension governs the approximation rate, with a network count bounded by where , and this extends to countably -rectifiable measures under an exponential-decay assumption on the components. The authors provide constructive space-filling schemes that couple a Lipschitz map approximant with a space-filling network to produce a push-forward closely approximating the target measure in Wasserstein distance. This yields universality results with explicit Lipschitz and architectural bounds, improves previous rates tied to ambient dimension, and demonstrates space-filling neural constructions for generating measures on low-dimensional geometric objects. The results have potential implications for synthetic data generation, geometric measure approximation, and learning measures supported on manifolds or unions of Lipschitz images.

Abstract

We derive universal approximation results for the class of (countably) -rectifiable measures. Specifically, we prove that -rectifiable measures can be approximated as push-forwards of the one-dimensional Lebesgue measure on using ReLU neural networks with arbitrarily small approximation error in terms of Wasserstein distance. What is more, the weights in the networks under consideration are quantized and bounded and the number of ReLU neural networks required to achieve an approximation error of is no larger than with . This result improves Lemma IX.4 in Perekrestenko et al. as it shows that the rate at which tends to infinity as tends to zero equals the rectifiability parameter , which can be much smaller than the ambient dimension. We extend this result to countably -rectifiable measures and show that this rate still equals the rectifiability parameter provided that, among other technical assumptions, the measure decays exponentially on the individual components of the countably -rectifiable support set.

Paper Structure

This paper contains 13 sections, 53 theorems, 289 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Notation
  3. Neural Networks
  4. Approximation of Lipschitz Functions
  5. Rectifiable Measures
  6. Generation of Rectifiable Measures
  7. Generation of Countably $m$-rectifiable Measures
  8. Approximations by Uniform Mixtures and Space-Filling Results
  9. Properties of Lipschitz Mappings
  10. Tools from General Measure Theory
  11. Properties of Wasserstein Distance
  12. Auxiliary Results
  13. Properties of the Sawtooth Function

Key Result

Lemma 1

Set $W_1(x)=(I_m, -I_m)^{\mathsf{T}}x$ for all $x\in\mathbb R\xspace^{m}$ and $W_2(x)=(I_m, -I_m)x$ for all $x\in\mathbb R\xspace^{2m}$ and consider the ReLU neural network $i_m\in \mathcal{N}\xspace_{m,m}$ defined according to $i_m=W_2\circ\rho\circ W_1$. Then, we have $i_m(x)=x$ for all $x\in\math

Figures (1)

  • Figure 1: Illustration of proof technique for the $m$-rectifiable measure $\nu$ supported on the $m$-rectifiable set $f(\mathcal{A}\xspace)$ on $\mathbb R\xspace^n$.

Theorems & Definitions (118)

  • Definition 1
  • Definition 2
  • Definition 3
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Definition 4
  • Lemma 3
  • Lemma 4
  • ...and 108 more