Network Design for Wafer-Scale Systems with Wafer-on-Wafer Hybrid Bonding

TL;DR

This paper proposes four reticle placements (Aligned, Interleaved, Rotated, and Contoured) that improve throughput by up to 250%, reduce latency by up to 36%, and decrease energy per transmitted byte by up to 38%.

Abstract

Transformer-based large language models are increasingly constrained by data movement as communication bandwidth drops sharply beyond the chip boundary. Wafer-scale integration using wafer-on-wafer hybrid bonding alleviates this limitation by providing ultra-high bandwidth between reticles on bonded wafers. In this paper, we investigate how the physical placement of reticles on wafers influences the achievable network topology and the resulting communication performance. Starting from a 2D mesh-like baseline, we propose four reticle placements (Aligned, Interleaved, Rotated, and Contoured) that improve throughput by up to 250%, reduce latency by up to 36%, and decrease energy per transmitted byte by up to 38%.

Figures (11)

• Figure 1: (§ \ref{['sec:opt']}) Different methods of building wafer-scale systems by optimizing the placement of reticles on the wafers.
• Figure 2: (§ \ref{['sec:evaluation-latency-throughput']}) Latency vs. Load for with 300 mm wafers and maximized utilization (permutation traffic).
• Figure 3: (§ \ref{['sec:evaluation-latency-throughput']}) Latency of Logic-on-Interconnect.
• Figure 4: (§ \ref{['sec:evaluation-latency-throughput']}) Latency of Logic-on-Logic.
• Figure 5: (§ \ref{['sec:evaluation-latency-throughput']}) Throughput of Logic-on-Interconnect.