Rigorous Error Analysis for Logarithmic Number Systems

TL;DR

This work addresses the lack of tight worst-case error bounds for Logarithmic Number Systems (LNS) used in high-dynamic-range computing. It develops a rigorous, parametric framework to bound errors arising from three core LNS techniques—first-order Taylor interpolation, error-correction, and co-transformation—across both $\Phi^+(x)$ and $\Phi^-(x)$ mappings, with explicit dependence on fixed-point epsilon $\epsilon$, table spacings $\Delta$, and related parameters. The authors prove and combine multiple lemmas and theorems to yield comprehensive bounds, then validate them via a Python implementation showing the bounds are tight and conservative. They also discuss adaptations to other bases and practical impacts for hardware/software design, including ROM-table sizing and proof-checking directions. Overall, the paper provides a principled, actionable foundation to design and verify LNS arithmetic with guaranteed accuracy bounds, reducing over-provisioning in HPC/ML hardware.

Abstract

Logarithmic Number Systems (LNS) hold considerable promise in helping reduce the number of bits needed to represent a high dynamic range of real-numbers with finite precision, and also efficiently support multiplication and division. However, under LNS, addition and subtraction turn into non-linear functions that must be approximated - typically using precomputed table-based functions. Additionally, multiple layers of error correction are typically needed to improve result accuracy. Unfortunately, previous efforts have not characterized the resulting error bound. We provide the first rigorous analysis of LNS, covering detailed techniques such as co-transformation that are crucial to implementing subtraction with reasonable accuracy. We provide theorems capturing the error due to table interpolations, the finite precision of pre-computed values in the tables, and the error introduced by fix-point multiplications involved in LNS implementations. We empirically validate our analysis using a Python implementation, showing that our analytical bounds are tight, and that our testing campaign generates inputs diverse-enough to almost match (but not exceed) the analytical bounds. We close with discussions on how to adapt our analysis to LNS systems with different bases and also discuss many pragmatic ramifications of our work in the broader arena of scientific computing and machine learning.

Figures (14)

• Figure 1: Plots of $\Phi^+(x)$ and $\Phi^-(x)$
• Figure 2: Details of defining $E_{\Delta}(\mathbf{i})$
• Figure 3: We do first-order Taylor approximation of $\Phi(x)$, obtaining $\hat{\Phi}_T$. This still has an error $E_T$. We plot $E^{+}_{T}$ here.
• Figure 4: Error ratio $P(x)$ in each $\Delta$ interval
• Figure 5: Co-transformation Illustration for the case where $x\in (-1, -\Delta_b)$: The positions of $r_c$,$r_{ab}$,$r_b$ and $r_a$ that are derived from this $x$ are shown. Here, $r_c$ is the closest index value of $T_c$ on the left of $x$. The gap of $x$ and $r_c$ is $-r_{ab}$. now $r_b$ is the closest index value of $T_b$ on the left of $r_{ab}$. Finally, the gap of $r_{ab}$ and $r_b$ is $-r_a$.

Theorems & Definitions (14)

• lemma 1
• lemma 2
• theorem 1
• lemma 3
• lemma 4
• lemma 5
• lemma 6
• lemma 7
• lemma 8
• lemma 9