Generalized Methodology for Determining Numerical Features of Hardware Floating-Point Matrix Multipliers: Part I

TL;DR

This work presents a generalized, architecture-agnostic methodology for determining the numerical features of hardware matrix-multiplier units, extending prior analyses to consumer GPUs. It defines a test-vector generation model, key parameters ($N_{FMA}$, $p_{in}$, $p_{out}$, $n_{eab}$, $n_{ecb}$), and iterative algorithms to uncover subnormal support, alignment and carry bits, FMA size, and rounding behavior across formats. Applied to RTX 3060 and Ada 1000, the approach reveals that consumer GPUs share many numerical features with data-center GPUs for several input formats and demonstrates the portability of the method across architectures; results are complemented by a GitHub release with CUDA and MATLAB tooling. The work supports standardization efforts and helps practitioners interpret cross-platform numerical results, with Part II planned to broaden coverage to more vendors and formats.

Abstract

Numerical features of matrix multiplier hardware units in NVIDIA and AMD data centre GPUs have recently been studied. Features such as rounding, normalisation, and internal precision of the accumulators are of interest. In this paper, we extend the methodology for analysing those features, to consumer-grade NVIDIA GPUs by implementing an architecture-independent test scheme for various input and output precision formats. Unlike current approaches, the proposed test vector generation method neither performs an exhaustive search nor relies on hard-coded {constants that are device-specific, yet remains applicable to a wide range of mixed-precision formats. We have applied the scheme to the RTX-3060 (Ampere architecture), and Ada RTX-1000 (Ada Lovelace architecture) graphics cards and determined numerical features of matrix multipliers for binary16, TensorFloat32, and bfloat16 input floating point formats and binary16 and binary32 IEEE 754 output formats. Our methodology allowed us to determine that} the numerical features of RTX-3060, a consumer-grade GPU, are identical to those of the A100, a data centre GPU. We do not expect our code to require any changes for performing analysis of matrix multipliers on newer NVIDIA GPUs, Hopper or Blackwell, and their future successors, and any input/output format combination, including the latest 8-bit floating-point formats.