Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RANDSMAPs: Random-Feature/multi-Scale Neural Decoders with Mass Preservation

Dimitrios G. Patsatzis, Alessandro Della Pia, Lucia Russo, Constantinos Siettos

Abstract

We introduce RANDSMAPs (Random-feature/multi-scale neural decoders with Mass Preservation), numerical analysis-informed, explainable neural decoders designed to explicitly respect conservation laws when solving the challenging ill-posed pre-image problem in manifold learning. We start by proving the equivalence of vanilla random Fourier feature neural networks to Radial Basis Function interpolation and the double Diffusion Maps (based on Geometric Harmonics) decoders in the deterministic limit. We then establish the theoretical foundations for RANDSMAP and introduce its multiscale variant to capture structures across multiple scales. We formulate and derive the closed-form solution of the corresponding constrained optimization problem and prove the mass preservation property. Numerically, we assess the performance of RANDSMAP on three benchmark problems/datasets with mass preservation obtained by the Lighthill-Whitham-Richards traffic flow PDE with shock waves, 2D rotated MRI brain images, and the Hughes crowd dynamics PDEs. We demonstrate that RANDSMAPs yield high reconstruction accuracy at low computational cost and maintain mass conservation at single-machine precision. In its vanilla formulation, the scheme remains applicable to the classical pre-image problem, i.e., when mass-preservation constraints are not imposed.

RANDSMAPs: Random-Feature/multi-Scale Neural Decoders with Mass Preservation

Abstract

We introduce RANDSMAPs (Random-feature/multi-scale neural decoders with Mass Preservation), numerical analysis-informed, explainable neural decoders designed to explicitly respect conservation laws when solving the challenging ill-posed pre-image problem in manifold learning. We start by proving the equivalence of vanilla random Fourier feature neural networks to Radial Basis Function interpolation and the double Diffusion Maps (based on Geometric Harmonics) decoders in the deterministic limit. We then establish the theoretical foundations for RANDSMAP and introduce its multiscale variant to capture structures across multiple scales. We formulate and derive the closed-form solution of the corresponding constrained optimization problem and prove the mass preservation property. Numerically, we assess the performance of RANDSMAP on three benchmark problems/datasets with mass preservation obtained by the Lighthill-Whitham-Richards traffic flow PDE with shock waves, 2D rotated MRI brain images, and the Hughes crowd dynamics PDEs. We demonstrate that RANDSMAPs yield high reconstruction accuracy at low computational cost and maintain mass conservation at single-machine precision. In its vanilla formulation, the scheme remains applicable to the classical pre-image problem, i.e., when mass-preservation constraints are not imposed.
Paper Structure (28 sections, 9 theorems, 108 equations, 19 figures, 10 tables)

This paper contains 28 sections, 9 theorems, 108 equations, 19 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Statement
  3. Manifold learning and encoding.
  4. The Pre-Image (decoding) problem.
  5. Problem statement.
  6. Numerical analysis-based decoders and conservation properties
  7. Decoding with POD/SVD
  8. Decoding with k-NN
  9. Decoding with DDM/GH
  10. The RANDSMAP decoder: Random-features Neural Decoder with Mass Preservation
  11. Vanilla Random Feature Decoders and their equivalence to the DDM decoder and RBFs interpolation in the deterministic limit
  12. Multi-scale random Fourier features
  13. RANDSMAPS: Random and multi-scale feature neural decoders with mass-conservation
  14. Numerical results
  15. Case Study 1: The Lighthill-Whitham-Richards 1D Traffic Model Dataset (M=400)
  16. ...and 13 more sections

Key Result

Proposition 1

Suppose the dataset $\mathcal{X}=\{x_i\}_{i=1}^N\subset\mathbb{R}^M$ is mass-preserving, i.e., the data matrix $X \in \mathbb{R}^{M \times N}$ satisfies the (normalized) sum-to-one constraint $1_M^\top X = 1_N^\top$. Let the matrix $\widehat{X}^* \in \mathbb{R}^{M \times L}$ contain the reconstructe where $V_r \in \mathbb{R}^{N \times r}$ and $\Lambda_r \in \mathbb{R}^{r \times r}$ denote matrices

Figures (19)

  • Figure 1: Schematic of the multi-scale Random Feature Neural Network decoder with random Fourier features; the features follow Eq. \ref{['eq:fourierrand']} with phase parameter $b_k \sim \mathcal{U}[0,2\pi)$ and frequency vectors $w_k$ sampled from the conditional probability distribution in Eq. \ref{['eq:conditionalprobab']}, i.e., sampling $\sigma_w \sim \mathcal{U}[a,b)$ and then draw $w \mid \sigma_w \sim \mathcal{N}(0, \sigma_w^2 I_d)$.
  • Figure 2: Reconstruction error vs. computational time for the LWR 1D traffic model dataset ($M=400$). Results are shown for $N=2000$ training points. Panel \ref{['fig:LWR_dec_tr']} shows mean relative $L_2$ error on the training set vs. training time. Panels \ref{['fig:LWR_dec_ts']} and \ref{['fig:LWR_dec_ts_linf']} show mean relative $L_2$ and $L_\infty$ errors, respectively, on the testing set vs. inference time. For the stochastic RANDSMAP decoders (RANDSMAP-RFF, RANDSMAP-MS-RFF, RANDSMAP-Sig) with $P=N,N/2,N/4$, points show the median over 100 random initializations and error bars indicate the 5–95% percentile range. Deterministic decoders (DDM, $k$-NN) are shown without error bars. Detailed numerical results are provided in Tables \ref{['tab:LWR_dec_tr_all']} and \ref{['tab:LWR_dec_ts_all']}.
  • Figure 3: Reconstruction error vs. computational time for the 2D rotated MRI images dataset ($M=128 \times 128$). Results are shown for $N=720$ training points. Panel \ref{['fig:MRI_dec_tr']} shows mean relative $L_2$ error on the training set vs. training time. Panels \ref{['fig:MRI_dec_ts']} and \ref{['fig:MRI_dec_ts_linf']} show mean $L_2$ and $L_\infty$ errors, respectively, on the testing set vs. inference time. For the stochastic RANDSMAP decoders (RANDSMAP-RFF, RANDSMAP-MS-RFF, RANDSMAP-Sig) with $P=N/4,N/2,N$, points show the median over 100 random initializations and error bars indicate the 5–95% percentile range. Deterministic decoders (DDM, $k$-NN) are shown without error bars. Detailed numerical results are provided in Tables \ref{['tab:MRI_dec_tr_all']} and \ref{['tab:MRI_dec_ts_all']}.
  • Figure 4: Reconstruction error vs. computational time for the Hughes 2D pedestrian model dataset ($M=200 \times 50$). Results are shown for $N=5000$ training points. Panel \ref{['fig:Hughes_dec_tr']} shows the mean relative $L_2$ error on the training set vs. training time. Panels \ref{['fig:Hughes_dec_ts']} and \ref{['fig:Hughes_dec_ts_linf']} show mean $L_2$ and $L_\infty$ errors, respectively, on the testing set vs. inference time. For the stochastic RANDSMAP decoders (RANDSMAP-RFF, RANDSMAP-MS-RFF, RANDSMAP-Sig) with $P=N/4,N/2,N$, points show the median over 100 random initializations and error bars indicate the 5–95% percentile range. Deterministic decoders (DDM, $k$-NN) are shown without error bars. Detailed numerical results are provided in Tables \ref{['tab:Hughes_dec_tr_all']} and \ref{['tab:Hughes_dec_ts_all']}.
  • Figure E.1: Reconstruction error vs. computational time for the Swiss Roll dataset ($M=3$). Results are shown for $N=1000$ (top row) and $N=2000$ (bottom row) training points. Panels \ref{['fig:SR_dec_tr1K']}, \ref{['fig:SR_dec_tr2K']} show the mean relative $L_2$ error $e_{2,i}$ (Eq. \ref{['eq:recon_errors']}) on the training set vs. training time. Panels \ref{['fig:SR_dec_ts1K']}, \ref{['fig:SR_dec_ts2K']} and \ref{['fig:SR_dec_ts1K_linf']}, \ref{['fig:SR_dec_ts2K_linf']} show the mean relative $L_2$ and $L_\infty$ error, $e_{2,i}$ and $e_{\infty,i}$, respectively, on the testing set vs. inference time. For the stochastic RFNN decoders (RFNN-RFF, RFNN-MS-RFF, RFNN-Sig) with $P=N,N/2,N/4$, points indicate the median over 100 runs; error bars show the 5–95% percentile range. Deterministic decoders (DDM, $k$-NN) are shown without error bars. Detailed numerical results are provided in Tables \ref{['tab:SR_dec_tr_all']} and \ref{['tab:SR_dec_ts_all']}.
  • ...and 14 more figures

Theorems & Definitions (25)

  • Definition 1: Manifold
  • Definition 2: Tangent space and tangent bundle to a manifold
  • Definition 3: Riemannian manifold and metric
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Remark 1
  • Theorem 1
  • proof
  • ...and 15 more