Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error Analysis of Matrix Multiplication Emulation Using Ozaki-II Scheme

Yuki Uchino, Katsuhisa Ozaki, Toshiyuki Imamura

TL;DR

The paper addresses the challenge of obtaining reliable high-precision matrix products via Ozaki-II, a CRT-based emulation that composes many low-precision GEMMs. It develops a rigorous deterministic error analysis, deriving explicit bounds that depend on the CRT moduli, scaling vectors, and input magnitude, and introduces two practical error estimates with different tightness and computational costs. The results explain how exponent distributions and the number of moduli influence accuracy and provide a foundation for automatic parameter tuning in Ozaki-II. The work advances the reliability and portability of CRT-based high-precision emulation on modern hardware, with potential extension to complex-valued matrix multiplication.

Abstract

The Ozaki-II scheme is an emulation method that leverages the Chinese Remainder Theorem to compute high-precision matrix multiplication via a sequence of low-precision matrix multiplications. In this scheme, the attainable numerical accuracy improves as the number of low-precision matrix multiplications increases. Previous numerical studies have shown that single- and double-precision matrix multiplication using the Ozaki-II scheme achieves higher throughput than that of standard BLAS routines on modern AI hardware equipped with fast INT8 matrix multiply-accumulate units with INT8 inputs and INT32 accumulation. However, the accuracy of the Ozaki-II scheme can degrade when the exponent distribution of the input matrices is wide, in which case a large number of low-precision matrix multiplications is required to obtain high-precision results. In this paper, we present a rigorous deterministic error analysis of the Ozaki-II scheme. The proposed analysis not only clarifies the accuracy behavior of the method but also enables the estimation of the number of low-precision matrix multiplications required to achieve a desired level of numerical accuracy.

Error Analysis of Matrix Multiplication Emulation Using Ozaki-II Scheme

TL;DR

The paper addresses the challenge of obtaining reliable high-precision matrix products via Ozaki-II, a CRT-based emulation that composes many low-precision GEMMs. It develops a rigorous deterministic error analysis, deriving explicit bounds that depend on the CRT moduli, scaling vectors, and input magnitude, and introduces two practical error estimates with different tightness and computational costs. The results explain how exponent distributions and the number of moduli influence accuracy and provide a foundation for automatic parameter tuning in Ozaki-II. The work advances the reliability and portability of CRT-based high-precision emulation on modern hardware, with potential extension to complex-valued matrix multiplication.

Abstract

The Ozaki-II scheme is an emulation method that leverages the Chinese Remainder Theorem to compute high-precision matrix multiplication via a sequence of low-precision matrix multiplications. In this scheme, the attainable numerical accuracy improves as the number of low-precision matrix multiplications increases. Previous numerical studies have shown that single- and double-precision matrix multiplication using the Ozaki-II scheme achieves higher throughput than that of standard BLAS routines on modern AI hardware equipped with fast INT8 matrix multiply-accumulate units with INT8 inputs and INT32 accumulation. However, the accuracy of the Ozaki-II scheme can degrade when the exponent distribution of the input matrices is wide, in which case a large number of low-precision matrix multiplications is required to obtain high-precision results. In this paper, we present a rigorous deterministic error analysis of the Ozaki-II scheme. The proposed analysis not only clarifies the accuracy behavior of the method but also enables the estimation of the number of low-precision matrix multiplications required to achieve a desired level of numerical accuracy.
Paper Structure (21 sections, 10 theorems, 97 equations, 4 figures, 3 algorithms)

This paper contains 21 sections, 10 theorems, 97 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Introduction
  3. Notation
  4. Related Work
  5. Long-Multiplication-Based Methods
  6. CRT-Based Methods
  7. Ozaki-II Scheme Using INT8 Matrix Engines
  8. Determination of Constants
  9. Determination of Scaling Vectors
  10. Matrix Multiplication via INT8
  11. Theoretical Results
  12. Proofs
  13. Properties
  14. Proof of Lemma \ref{['lem0']}
  15. Proof of Lemma \ref{['lem1']}
  16. ...and 6 more sections

Key Result

Theorem 1

Let $x \in \mathbb{Z}$. Suppose that $p_1,\dots,p_N \in \mathbb{N}_{\ge 2}$ are pairwise coprime integers and $\mathcal{P} := \prod_{1 \le \ell \le N}{p_\ell}$. For $\ell=1,\dots,N$, define $q_\ell \in \mathbb{N}$ as modular multiplicative inverses of $\mathcal{P}/p_\ell$ (i.e., $\mathcal{P}/p_\ell Then, it holds that

Figures (4)

  • Figure 1: Diagram of $s_{\ell 1}$ and $s_{\ell 2}$.
  • Figure 2: Diagram of $Q$, $\Delta_{Q_1}$, and $\Delta_{Q_2}$.
  • Figure 3: Comparison between theoretical error bounds and observed numerical errors for DGEMM emulation. est_max and est_min are the maximum and minimum error bounds derived from Theorem \ref{['thm:err']} using \ref{['thm:err-1']}, while est2_max and est2_min are those obtained from \ref{['thm:err-2']}. err_max and err_min are the maximum and minimum values of the actual emulation error, respectively. For reference, err64_max is the maximum error of native DGEMM.
  • Figure 4: Comparison between theoretical error bounds and observed numerical errors for SGEMM emulation. The meanings of the plotted lines are analogous to those in Fig. \ref{['fig:comparison_d']}, with err32_max being the maximum error of native SGEMM.

Theorems & Definitions (16)

  • Theorem 1: Chinese Remainder Theorem
  • Theorem 2
  • Lemma 1: Uniqueness of candidate of $A'B'$
  • Lemma 2: Truncation error
  • Lemma 3: Accumulation error
  • Lemma 4: Validity of $Q$
  • Lemma 5: Error in final reduction
  • Lemma 6: Definition of floating-point numbers
  • Lemma 7: rump2008accuratejeannerod2018relative
  • Lemma 8: Jeannerod2013improved
  • ...and 6 more