Speeding up Monte Carlo Integration: Control Neighbors for Optimal Convergence

TL;DR

The paper tackles the slow convergence of standard Monte Carlo integration by introducing control neighbors, a variance-reduction scheme that uses leave-one-out 1-NN predictions as control variates on metric spaces. This yields a provably faster rate $\mathcal{O}(n^{-1/2} n^{-s/d})$ for Hölder integrands with regularity $s\in(0,1]$, and is applicable beyond Euclidean cubes to manifolds and other compact spaces. The authors provide a comprehensive theoretical framework (ball-measure distributions, NN-distance moments, RMSE and concentration bounds) and show practical implementation details, with numerical experiments across diverse spaces and an OT application demonstrating substantial variance reduction. The method maintains a simple linear-in-phi form, does not require known integrals of the control variates, and can be fitted after sampling, making it attractive for repeated-integral tasks and manifold-valued domains. Overall, control neighbors offer a principled, broadly applicable approach to accelerate Monte Carlo integration in smooth regimes where evaluations are costly.

Abstract

A novel linear integration rule called $\textit{control neighbors}$ is proposed in which nearest neighbor estimates act as control variates to speed up the convergence rate of the Monte Carlo procedure on metric spaces. The main result is the $\mathcal{O}(n^{-1/2} n^{-s/d})$ convergence rate -- where $n$ stands for the number of evaluations of the integrand and $d$ for the dimension of the domain -- of this estimate for Hölder functions with regularity $s \in (0,1]$, a rate which, in some sense, is optimal. Several numerical experiments validate the complexity bound and highlight the good performance of the proposed estimator.

Figures (7)

• Figure 1: Voronoï cells and Delaunay triangulations on $[0,1]^2$ and $\mathbb{S}^2$. (Made with Python package Matplotlibhunter2007matplotlib)
• Figure 2: Root mean squared errors obtained over $100$ replications for functions $\varphi_1$ (top) and $\varphi_2$ (bottom) in Eq. \ref{['eq:phi1phi2']} in dimension $d \in \{2,3,4\}$ (left to right).
• Figure 3: Root mean squared errors (left) and boxplots of the errors (right) obtained over $100$ replications for integrands $X \mapsto \operatorname{tr}(X)^k$ for $k \in \{1,2\}$ in Eq. \ref{['eq:Odphik']} with respect to the unit Haar measure on the orthogonal group $O_3(\mathbb{R})$.
• Figure 4: Root mean squared errors (top) and boxplots of the errors (bottom) obtained over $100$ replications for functions $\varphi_3$ (left), $\varphi_4$ (middle) and $\varphi_5$ (right) in Eq. \ref{['eq:sphereintegrands']} when integrating over $\mathbb{S}^2$.
• Figure 5: Boxplots of Sliced-Wasserstein estimates SW-MC and SW-CVNN for Gaussian distributions on $\mathbb{R}^q$ with $q \in \{3;6\}$. The boxplots are obtained over $100$ replications.

Theorems & Definitions (22)

• Definition 1: Nearest neighbors and distances
• Definition 2: Voronoi cells and volumes
• Definition 3: Leave-one-out neighbors, Voronoi cells and volumes
• Lemma 1
• Definition 4: Degree and cumulative volume
• Lemma 2
• Proposition 1: Quadrature rules
• Remark 1
• Remark 2
• Theorem : Corollary 2.2 in kokarev2021