Neural Low-Discrepancy Sequences

TL;DR

This work addresses the limitation of existing low-discrepancy constructions that either produce fixed-$N$ sets or cannot extend to sequences. It introduces NeuroLDS, a neural, index-based generator that maps sequence indices to points in $[0,1]^d$ via a sinusoidal encoding and a deep MLP, trained in two stages: first to align with a classical LD sequence using $L_2$-based loss, then to minimize prefix discrepancies across all prefixes. NeuroLDS consistently achieves lower prefix discrepancies than traditional LD constructions across multiple discrepancy measures and demonstrates practical gains in quasi-MMonte Carlo integration, robotic motion planning, and scientific machine learning. The approach offers a flexible, extensible framework for neural design of LDS, with potential extensions to broader discrepancy notions like Stein discrepancies and other problem-specific weightings, while reaffirming the value of classical pre-alignment for stability.

Abstract

Low-discrepancy points are designed to efficiently fill the space in a uniform manner. This uniformity is highly advantageous in many problems in science and engineering, including in numerical integration, computer vision, machine perception, computer graphics, machine learning, and simulation. Whereas most previous low-discrepancy constructions rely on abstract algebra and number theory, Message-Passing Monte Carlo (MPMC) was recently introduced to exploit machine learning methods for generating point sets with lower discrepancy than previously possible. However, MPMC is limited to generating point sets and cannot be extended to low-discrepancy sequences (LDS), i.e., sequences of points in which every prefix has low discrepancy, a property essential for many applications. To address this limitation, we introduce Neural Low-Discrepancy Sequences ($NeuroLDS$), the first machine learning-based framework for generating LDS. Drawing inspiration from classical LDS, we train a neural network to map indices to points such that the resulting sequences exhibit minimal discrepancy across all prefixes. To this end, we deploy a two-stage learning process: supervised approximation of classical constructions followed by unsupervised fine-tuning to minimize prefix discrepancies. We demonstrate that $NeuroLDS$ outperforms all previous LDS constructions by a significant margin with respect to discrepancy measures. Moreover, we demonstrate the effectiveness of $NeuroLDS$ across diverse applications, including numerical integration, robot motion planning, and scientific machine learning. These results highlight the promise and broad significance of Neural Low-Discrepancy Sequences. Our code can be found at https://github.com/camail-official/neuro-lds.

Figures (7)

• Figure 1: Schematic of the proposed NeuroLDS model. First, each index $i \in \{1,\dots,N\}$ is mapped to a sinusoidal feature vector $\psi_i \in \mathbb{R}^{1+2K}$. Second, the encoded features are passed through an $L$-layer multilayer perceptron (MLP), which outputs $\mathbf{X}_i \in [0,1]^d$. Finally, the collection $\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$ forms a (learned) low-discrepancy sequence within the unit hypercube.
• Figure 2: Comparison plots of different discrepancy metrics for 4d point sequences up to a length of 10000 as a function of sequence length: $D_{2}^{\text{sym}}$ (left), $D_{2}^{\text{star}}$ (middle), and $D_{2}^{\text{ctr}}$ (right). Each panel compares Sobol', Halton, NeuroLDS, and Scrambled Sobol' (mean over 32 scrambled sequences).
• Figure 3: Sobol' indices for the Borehole function.
• Figure 4: Visualization of kinematic chain task for a 4d chain in a semi-circular tunnel.
• Figure 5: First $N \in \{64,128,256\}$ points of Sobol' (blue) and NeuroLDS (orange) in $2$d. NeuroLDS was trained with the $D_{2}^{\text{star}}$ discrepancy and Optuna-tuned hyperparameters. Sobol' exhibits visible structure and clustering, while NeuroLDS distributes points more irregularly yet evenly, mitigating alignment artifacts and leading to lower prefix discrepancy.