Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Batch Method with Momentum Correction

Yanshun Zhao, Jingrun Chen, Zhiwen Zhang

TL;DR

The paper tackles the computational bottleneck of simulating large interacting particle systems with potentially singular kernels by extending the Random Batch Method with Momentum Correction (RBM-M). It provides a theoretical error bound that scales with $(1-β^2)√τ$ (up to small terms) and demonstrates asymptotic unbiasedness with reduced variance, supported by numerical experiments showing improved accuracy for highly singular kernels while maintaining $O(N)$ cost. The results broaden the practical applicability of RBM to rough kernels and offer insights into parameter choices and kernel regularization. The work suggests promising future directions, including adaptive momentum parameters and momentum-based variants akin to Adam or RMSProp for further robustness and efficiency.

Abstract

The Random Batch Method (RBM) is an effective technique to reduce the computational complexity when solving certain stochastic differential problems (SDEs) involving interacting particles. It can transform the computational complexity from O(N^2) to O(N), where N represents the number of particles. However, the traditional RBM can only be effectively applied to interacting particle systems with relatively smooth kernel functions to achieve satisfactory results. To address the issue of non-convergence of the RBM in particle interaction systems with significant singularities, we propose some enhanced methods to make the modified algorithm more applicable. The idea for improvement primarily revolves around a momentum-like correction, and we refer to the enhanced algorithm as the Random Batch Method with Momentum Correction ( RBM-M). We provide a theoretical proof to control the error of the algorithm, which ensures that under ideal conditions it has a smaller error than the original algorithm. Finally, numerical experiments have demonstrated the effectiveness of the RBM-M algorithm.

Random Batch Method with Momentum Correction

TL;DR

The paper tackles the computational bottleneck of simulating large interacting particle systems with potentially singular kernels by extending the Random Batch Method with Momentum Correction (RBM-M). It provides a theoretical error bound that scales with (up to small terms) and demonstrates asymptotic unbiasedness with reduced variance, supported by numerical experiments showing improved accuracy for highly singular kernels while maintaining cost. The results broaden the practical applicability of RBM to rough kernels and offer insights into parameter choices and kernel regularization. The work suggests promising future directions, including adaptive momentum parameters and momentum-based variants akin to Adam or RMSProp for further robustness and efficiency.

Abstract

The Random Batch Method (RBM) is an effective technique to reduce the computational complexity when solving certain stochastic differential problems (SDEs) involving interacting particles. It can transform the computational complexity from O(N^2) to O(N), where N represents the number of particles. However, the traditional RBM can only be effectively applied to interacting particle systems with relatively smooth kernel functions to achieve satisfactory results. To address the issue of non-convergence of the RBM in particle interaction systems with significant singularities, we propose some enhanced methods to make the modified algorithm more applicable. The idea for improvement primarily revolves around a momentum-like correction, and we refer to the enhanced algorithm as the Random Batch Method with Momentum Correction ( RBM-M). We provide a theoretical proof to control the error of the algorithm, which ensures that under ideal conditions it has a smaller error than the original algorithm. Finally, numerical experiments have demonstrated the effectiveness of the RBM-M algorithm.

Paper Structure

This paper contains 17 sections, 9 theorems, 107 equations, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Interacting particle systems
  3. Related work
  4. Advantage of Algorithm
  5. RBM-M
  6. The deficiency of RBM
  7. Regularization of kernel
  8. Correct algorithm with momentum
  9. Theoretical analysis of RBM-M
  10. The error of RBM-M
  11. Good statistical properties about variance and expectations
  12. The error estimation of rescaled relative entropy
  13. Numerical experiment
  14. The error is proportional to $\sqrt{\tau}$
  15. Compare the error between RBM-M and RBM
  16. ...and 2 more sections

Key Result

Proposition 2.1

Suppose $V$ is strongly convex on $\mathbb{R}^d$ so that $x \mapsto V(x) - \frac{1}{2}|x|^2$ is convex, and $\nabla V$, $\nabla^2 V$ have polynomial growth (i.e., $|\nabla V(x)| + |\nabla^2 V(x)| \leq C(1 + |x|^q)$ for some $q > 0$). Assume $K(\cdot)$ is bounded, Lipschitz on $\mathbb{R}^d$ with Lip

Figures (7)

  • Figure 1: RBM-M algorithm diagram.
  • Figure 2: Two order system--x.
  • Figure 3: Two order system--v.
  • Figure 4: A case of large singularity.
  • Figure 5: The steepness of different $\alpha$.
  • ...and 2 more figures

Theorems & Definitions (15)

  • Proposition 2.1
  • Remark 2.1
  • Remark 2.2
  • Proposition 3.1
  • Theorem 3.2
  • proof
  • Proposition 3.3
  • Theorem 3.4
  • proof
  • Theorem 3.5
  • ...and 5 more