Rounding error using low precision approximate random variables

TL;DR

The paper analyzes rounding errors when solving stochastic differential equations with the Euler–Maruyama scheme using low-precision approximate random variables. It derives a leading-order error model with two contributions, $\eta$ (zero-mean martingale-like) and $\eta'$ (smaller, possibly nonzero-mean), and shows how this model interacts with nested multilevel Monte Carlo (MLMC) incorporating approximate randomness. The authors establish a bound, $\mathbb{E}(|\widehat{X}_N - \overline{X}_N|^2) \leq C N ρ^2$, and demonstrate how precision and Kahan compensated summation affect two- and four-way differences, guiding practical hardware choices. Their results indicate that single precision offers substantial speedups across most levels, while half precision delivers the best gains on the coarsest levels, with Kahan summation further extending applicability; overall, the framework enables fast, accurate stochastic simulations by coupling low-precision arithmetic, approximate randomness, and nested MLMC.

Abstract

For numerical approximations to stochastic differential equations using the Euler-Maruyama scheme, we propose incorporating approximate random variables computed using low precisions, such as single and half precision. We propose and justify a model for the rounding error incurred, and produce an average case error bound for two and four way differences, appropriate for regular and nested multilevel Monte Carlo estimations. By considering the variance structure of multilevel Monte Carlo correction terms in various precisions with and without a Kahan compensated summation, we compute the potential speed ups offered from the various precisions. We find single precision offers the potential for approximate speed improvements by a factor of 7 across a wide span of discretisation levels. Half precision offers comparable improvements for several levels of coarse simulations, and even offers improvements by a factor of 10-12 for the very coarsest few levels.

Figures (5)

• Figure 1: A piecewise linear approximation of the Gaussian distribution's inverse cumulative distribution function, and the resultant probability density function.
• Figure 2: Rounding to the nearest even. We denote even representable values using "$\bullet$", odd values using "$\bigcirc$". Arrows show the values rounded to.
• Figure 3: The variance of the difference between the exact Euler-Maruyama estimate and the estimate using low precision approximate random variables from a piecewise cubic approximation with different precisions, corresponding to lemma \ref{['lemma:rounding_error_two_way']}. The precisions use differing numbers of bits for the mantissa. ($\blacklozenge$) 7 bits. ($\blacksquare$) 10 bits. ($\bullet$) 16 bits. ($\blacktriangledown$) 23 bits.
• Figure 4: The variance reduction when switching to approximate random variables with a nested multilevel Monte Carlo framework, showing the variances of various two and four way differences. The key indicates the difference, precision, and whether Kahan compensated summation is used.
• Figure 5: The potential savings from a nested multilevel Monte Carlo framework using approximate random variables from a piecewise linear approximation for various discretisation levels. ($\blacktriangledown$) Single precision. ($\blacksquare$) Half precision without Kahan compensated summation. ($\blacklozenge$) Half precision with Kahan compensated summation.

Theorems & Definitions (14)

• Lemma 2.1
• proof
• Lemma 2.2
• proof
• Lemma 2.3
• proof
• Lemma 2.4
• proof
• Lemma 2.5
• proof