Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bounds on non-linear errors for variance computation with stochastic rounding

El-Mehdi El Arar, Devan Sohier, Pablo de Oliveira Castro, Eric Petit

TL;DR

The study shows that for pairwise summation, using stochastic rounding using SR results in a probabilistic bound of the forward error proportional to log(n)u rather than the deterministic bound in O(log( n)u) when using the default rounding mode.

Abstract

The main objective of this work is to investigate non-linear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: the first is based on a bound of the variance and Bienaym{é}-Chebyshev inequality, while the second is based on martingales and Azuma-Hoeffding inequality. The study shows that for pairwise summation, using SR results in a probabilistic bound of the forward error proportional to log(n)u rather than the deterministic bound in O(log(n)u) when using the default rounding mode. We examine two algorithms that compute the variance, called ''textbook'' and ''two-pass'', which both exhibit non-linear errors. Using the two methods mentioned above, we show that these algorithms' forward errors have probabilistic bounds under SR in O($\sqrt$ nu) instead of nu for the deterministic bounds. We show that this advantage holds using pairwise summation for both textbook and two-pass, with probabilistic bounds of the forward error proportional to log(n)u.

Bounds on non-linear errors for variance computation with stochastic rounding

TL;DR

The study shows that for pairwise summation, using stochastic rounding using SR results in a probabilistic bound of the forward error proportional to log(n)u rather than the deterministic bound in O(log( n)u) when using the default rounding mode.

Abstract

The main objective of this work is to investigate non-linear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: the first is based on a bound of the variance and Bienaym{é}-Chebyshev inequality, while the second is based on martingales and Azuma-Hoeffding inequality. The study shows that for pairwise summation, using SR results in a probabilistic bound of the forward error proportional to log(n)u rather than the deterministic bound in O(log(n)u) when using the default rounding mode. We examine two algorithms that compute the variance, called ''textbook'' and ''two-pass'', which both exhibit non-linear errors. Using the two methods mentioned above, we show that these algorithms' forward errors have probabilistic bounds under SR in O( nu) instead of nu for the deterministic bounds. We show that this advantage holds using pairwise summation for both textbook and two-pass, with probabilistic bounds of the forward error proportional to log(n)u.
Paper Structure (25 sections, 17 theorems, 79 equations, 3 figures, 1 table)

This paper contains 25 sections, 17 theorems, 79 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notations and definitions
  3. Notations
  4. Floating-point background
  5. Stochastic rounding
  6. Pairwise summation
  7. BC method
  8. AH method
  9. Bias analysis
  10. Textbook algorithm
  11. Two-pass algorithm
  12. Error analysis for algorithms with non-linear error
  13. BC method
  14. Textbook algorithm
  15. Two-pass algorithm
  16. ...and 10 more sections

Key Result

Lemma 2.1

\newlabellem:proba0 Let $X$ and $Y$ two random variables, $a,b\in \mathbb{R}_+^*$, and $\lambda, \mu \in ]0; 1[$ such that: $\mathbb{P}(\left\vert X \right\vert\leq a) \geq 1- \lambda$ and $\mathbb{P}(\left\vert Y \right\vert\leq b) \geq 1- \mu$. Then

Figures (3)

  • Figure 1: SR-nearness.
  • Figure 1: Probabilistic error bounds over $n$ with probability $1- \lambda =0.9$ (left) and over $\lambda$ with $n=10^6$ (right) vs deterministic bound for the textbook algorithm.
  • Figure 2: The forward errors of textbook and two-pass algorithms in binary32 precision for floating-points chosen uniformly at random in $[-1;1]$ (left) and $[1024;1025]$ (right).

Theorems & Definitions (35)

  • Lemma 2.1
  • Proof 1
  • Lemma 2.2
  • Lemma 3.1
  • Theorem 3.2
  • Proof 2
  • Definition 3.3
  • Lemma 3.4: Azuma-Hoeffding inequality
  • Theorem 3.5
  • Proof 3
  • ...and 25 more