Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiword matrix multiplication over large finite fields in floating-point arithmetic

Jérémy Berthomieu, Stef Graillat, Dimitri Lesnoff, Theo Mary

TL;DR

This paper tackles the problem of efficiently computing $C=AB \bmod p$ over large primes $p$ using floating-point arithmetic, aiming to leverage highly optimized BLAS on modern CPUs and GPUs. It introduces multiword decompositions, where $A$ and $B$ are represented as sums of $u$ and $v$ words with carefully chosen scaling, and proves exactness under explicit bounds; it analyzes the trade-offs and derives a readable condition that guides the choice of $(u,v)$ and block size. The authors present a suite of algorithms, including a concatenated variant, and provide thorough theoretical guarantees and a comprehensive set of performance benchmarks on CPU and GPU architectures. The results show that primes up to about $2^{52}$ can be efficiently handled with multiword methods, outperforming the traditional single-word approach for primes beyond about 23 bits and offering advantages over multimodular CRT-based methods in many settings, especially for unbalanced matrices and large-scale problems.

Abstract

This article is concerned with the efficient computation of modular matrix multiplication C=AB mod p, a key kernel in computer algebra. We focus on floating-point arithmetic, which allows for using efficient matrix multiplication libraries. However, the existing approach is limited to primes p with bitsize at most half the mantissa size (e.g., 26 bits with double precision arithmetic), and becomes quite inefficient when p approaches this limit. We present a new approach that overcomes this limitation and can efficiently handle primes with larger bitsizes. The key idea is to use multiword decompositions, which represent A and B as scaled sums of u and v matrices (words) with smaller coefficients. We provide a rigorous analysis that proves the correctness of this approach for suitably chosen scaling parameters. Our analysis determines the maximum bitsize of p that can be handled for a given number of words; in particular, we show that decomposing in two words each input suffices to handle bitsizes almost equal to the full mantissa size (e.g., the 26 bits limit is raised to 52 bits in double precision arithmetic). Moreover, we show that (1,v) decompositions with v>1 are also of interest to handle intermediate bitsizes. We perform an extensive experimental analysis for various matrix shapes and prime bitsizes. Our performance benchmarks on both CPU and GPU architectures confirm the efficiency of the proposed approach, which can outperform the existing single word approach for bitsizes as low as 23, and can handle bitsizes as high as 52 while retaining high performance.

Multiword matrix multiplication over large finite fields in floating-point arithmetic

TL;DR

This paper tackles the problem of efficiently computing over large primes using floating-point arithmetic, aiming to leverage highly optimized BLAS on modern CPUs and GPUs. It introduces multiword decompositions, where and are represented as sums of and words with carefully chosen scaling, and proves exactness under explicit bounds; it analyzes the trade-offs and derives a readable condition that guides the choice of and block size. The authors present a suite of algorithms, including a concatenated variant, and provide thorough theoretical guarantees and a comprehensive set of performance benchmarks on CPU and GPU architectures. The results show that primes up to about can be efficiently handled with multiword methods, outperforming the traditional single-word approach for primes beyond about 23 bits and offering advantages over multimodular CRT-based methods in many settings, especially for unbalanced matrices and large-scale problems.

Abstract

This article is concerned with the efficient computation of modular matrix multiplication C=AB mod p, a key kernel in computer algebra. We focus on floating-point arithmetic, which allows for using efficient matrix multiplication libraries. However, the existing approach is limited to primes p with bitsize at most half the mantissa size (e.g., 26 bits with double precision arithmetic), and becomes quite inefficient when p approaches this limit. We present a new approach that overcomes this limitation and can efficiently handle primes with larger bitsizes. The key idea is to use multiword decompositions, which represent A and B as scaled sums of u and v matrices (words) with smaller coefficients. We provide a rigorous analysis that proves the correctness of this approach for suitably chosen scaling parameters. Our analysis determines the maximum bitsize of p that can be handled for a given number of words; in particular, we show that decomposing in two words each input suffices to handle bitsizes almost equal to the full mantissa size (e.g., the 26 bits limit is raised to 52 bits in double precision arithmetic). Moreover, we show that (1,v) decompositions with v>1 are also of interest to handle intermediate bitsizes. We perform an extensive experimental analysis for various matrix shapes and prime bitsizes. Our performance benchmarks on both CPU and GPU architectures confirm the efficiency of the proposed approach, which can outperform the existing single word approach for bitsizes as low as 23, and can handle bitsizes as high as 52 while retaining high performance.
Paper Structure (20 sections, 7 theorems, 37 equations, 7 figures, 4 tables)

This paper contains 20 sections, 7 theorems, 37 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Existing single word algorithm and its limitations
  3. Modular reductions in floating-point arithmetic
  4. Block matrix product
  5. New multiword algorithms
  6. Multiword matrix decomposition
  7. Multiword matrix multiplication
  8. Discussion on the cost of the algorithms
  9. Performance benchmarks
  10. Experimental setting
  11. Discussion of the results
  12. Square matrices
  13. Unbalanced matrices and effect of concatenation
  14. Summary: variant selection
  15. Comparison with the state-of-the-art
  16. ...and 5 more sections

Key Result

Proposition 1

\newlabelprop:fpModularReductionFMA0 alg:fpModularReductionFMA is correct for any integer input $x\in\mathbb{F}$ and a modulus $p$ such that $4 \leq p < 2^{t-1}$ and $x \leq 2^{t-2}p$.

Figures (7)

  • Figure 1: Performance benchmark for square matrices on CPU.
  • Figure 1: Number of products required by the multimodular and multiword approaches. \newlabelfig.nbprod-comparison0
  • Figure 2: Performance benchmark for square matrices on GPU.
  • Figure 3: Performance benchmark for unbalanced matrices on CPU.
  • Figure 4: Performance benchmark for unbalanced matrices on CPU, with concatenation.
  • ...and 2 more figures

Theorems & Definitions (14)

  • Proposition 1
  • Proof 1
  • Proposition 2
  • Proof 2
  • Proposition 3
  • Proof 3
  • Corollary 4
  • Proof 4
  • Lemma 1
  • Proof 5
  • ...and 4 more