Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Error Analysis of Limited-Precision Stochastic Rounding: Horner's Algorithm and Pairwise Summation

El-Mehdi El Arar, Massimiliano Fasi, Silviu-Ioan Filip, Mantas Mikaitis

Abstract

Stochastic rounding (SR) is a probabilistic rounding mode that mitigates errors in large-scale numerical computations, especially when prone to stagnation effects. Beyond numerical analysis, SR has shown significant benefits in practical applications such as deep learning and climate modelling. The definition of classical SR requires that results of arithmetic operations are known with infinite precision. This is often not possible, and when it is, the resulting hardware implementation can become prohibitively expensive in terms of energy, area, and latency. A more practical alternative is limited-precision SR, which only requires that the outputs of arithmetic operations are available in higher, finite, precision. We extend previous work on limited-precision SR presented in [El Arar et al., SIAM J. Sci. Comput. 47(5) (2025), B1227-B1249], which developed a framework to evaluate the trade-off between accuracy and hardware resource cost in SR implementations. Within this framework, we study the Horner algorithm and pairwise summation, providing both theoretical insights and practical experiments in these settings when using limited-precision SR.

Probabilistic Error Analysis of Limited-Precision Stochastic Rounding: Horner's Algorithm and Pairwise Summation

Abstract

Stochastic rounding (SR) is a probabilistic rounding mode that mitigates errors in large-scale numerical computations, especially when prone to stagnation effects. Beyond numerical analysis, SR has shown significant benefits in practical applications such as deep learning and climate modelling. The definition of classical SR requires that results of arithmetic operations are known with infinite precision. This is often not possible, and when it is, the resulting hardware implementation can become prohibitively expensive in terms of energy, area, and latency. A more practical alternative is limited-precision SR, which only requires that the outputs of arithmetic operations are available in higher, finite, precision. We extend previous work on limited-precision SR presented in [El Arar et al., SIAM J. Sci. Comput. 47(5) (2025), B1227-B1249], which developed a framework to evaluate the trade-off between accuracy and hardware resource cost in SR implementations. Within this framework, we study the Horner algorithm and pairwise summation, providing both theoretical insights and practical experiments in these settings when using limited-precision SR.
Paper Structure (8 sections, 3 theorems, 32 equations, 4 figures)

This paper contains 8 sections, 3 theorems, 32 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Stochastic Rounding
  3. Horner's algorithm
  4. Pairwise summation
  5. Numerical experiments
  6. Horner's algorithm
  7. Pairwise summation
  8. Conclusion

Key Result

lemma 1

Let $\delta_1, \delta_2,\ldots, \delta_n$ be random errors produced by a sequence of elementary operations using $\textup{SR}_{p,r}$, and let $\beta_1, \beta_2,\ldots, \beta_n$ be their corresponding errors incurred by $\mathop{\mathrm{fl}}\nolimits_{p+r}$. Then, the random variables $\alpha_k = \de Moreover, for all $1\leq i \leq n$, where $\mathcal{I}_i = \{k \in \mathbb{N} : i \le k \le n\}$ i

Figures (4)

  • Figure 1: Quantities used in the definitions \ref{['eq:sr']} and \ref{['eq:sr-imp']}.
  • Figure 2: Relative error of RN and $\text{SR}_{11,r}$ in IEEE 754 binary16 arithmetic ($p=11$). The coefficients are drawn from a uniform distribution over $[0,1]$ (left) and $[-1,1]$ (right).
  • Figure 3: The analogous experiment to \ref{['fig:fp-experiments']} for bfloat16 arithmetic ($p=8$).
  • Figure 4: Relative error of RN and $\text{SR}_{8,r}$ in IEEE-754 bfloat16 arithmetic ($p=8$) for pairwise summation. The floating point values are drawn from a uniform distribution over $[0,10^5]$ (left) and $[-10^5,10^5]$ (right).

Theorems & Definitions (12)

  • definition 1: Stochastic rounding
  • definition 2: limited-precision stochastic rounding
  • remark 1
  • lemma 1: effm25
  • theorem 1
  • proof
  • remark 2
  • remark 3
  • theorem 2
  • proof
  • ...and 2 more