MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products

TL;DR

MXDOTP addresses the need for efficient MX dot products on RISC-V by introducing a dedicated dot-product-accumulate unit and a four-operand instruction. It integrates with the Snitch core via SSRs to sustain high throughput while streaming per-block scales, achieving up to 356 GFLOPS/W for MXFP8 matmuls. On an eight-core 12 nm FinFET Snitch cluster, MXDOTP delivers 25x speedup and 12.5x energy efficiency improvements over a software MX baseline, with only a moderate area overhead. This work demonstrates that native hardware support for microscaling formats is essential to unlock their performance and energy benefits in AI workloads and proves that an ISA-extended approach can provide flexible, high-performance MX acceleration.

Abstract

Fast and energy-efficient low-bitwidth floating-point (FP) arithmetic is essential for Artificial Intelligence (AI) systems. Microscaling (MX) standardized formats have recently emerged as a promising alternative to baseline low-bitwidth FP formats, offering improved accuracy with a block-wise shared exponent scale combined with per-element values. However, efficiently executing the key linear algebra primitives for AI applications on MX formats requires specialized hardware support for the fundamental operators such as scaled dot product. In this work, we propose MXDOTP, the first RISC-V ISA extension for MX dot products, focusing on the 8-bit MXFP8 FP format. We extend the open-source Snitch RISC-V core with a dedicated MXFP8 dot product-accumulate unit, which fully consumes blocks of eight 8-bit operands packed into 64-bit inputs. To feed MXDOTP at full utilization with four operands per cycle, including block scales, we exploit Snitch's Stream Semantic Registers (SSRs), achieving up to 80% utilization with minimal impact on the Snitch core's architecture and no modification to the register file. Implemented in 12 nm FinFET, a cluster with eight MXDOTP-extended cores reaches up to 356 GFLOPS/W when computing MXFP8 matrix multiplications at 0.8 V, 1 GHz. Compared to a software baseline, where MX dot products are computed by type casting FP8 inputs to FP32 for higher accumulation precision and applying explicit block scaling, the cluster achieves 25x speedup and 12.5x better energy efficiency at a minimal 5.1% area increase.

Figures (7)

• Figure 1: (a) Simplified MXDOTP datapath (sign bits omitted). (b) Integration of MXDOTP into the Snitch core. (c) Execution of MXDOTP over multiple cycles with three pipeline stages. enable efficient operand streaming, ensuring full utilization without stalls.
• Figure 3: Area breakdown of the MXDOTP-extended Snitch core.
• Figure 4: (a) Throughput and (b) Energy Efficiency of FP32, FP8-to-FP32, and MXFP8mm kernels for varying inner dimensions, with rows and columns fixed at 64. FP32 does not fit into L1 with inner dimension of 256.
• Figure : FP32
• Figure : FP32