Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MCMComm: Hardware-Software Co-Optimization for End-to-End Communication in Multi-Chip-Modules

Ritik Raj, Shengjie Lin, William Won, Tushar Krishna

TL;DR

MCMComm tackles the end-to-end communication bottlenecks in multi-chip-module accelerators by introducing a congestion-aware, packaging-adaptive analytical framework. It couples end-to-end latency and energy-delay product modeling with hardware-software co-optimization, including diagonal links, on-package redistribution, and fine-grained pipelining. Scheduling is solved via two approaches—Genetic Algorithm (GA) and Mixed Integer Quadratic Programming (MIQP)—demonstrating up to 1.58x EdP improvement with GA and 2.7x with MIQP for CNNs and Vision Transformers, respectively. The work provides a practical pathway to scalable, energy-efficient MCM-based AI accelerators, accounting for memory type, packaging, and interconnect characteristics across diverse workloads.

Abstract

Increasing AI computing demands and slowing transistor scaling have led to the advent of Multi-Chip-Module (MCMs) based accelerators. MCMs enable cost-effective scalability, higher yield, and modular reuse by partitioning large chips into smaller chiplets. However, MCMs come at an increased communication cost, which requires critical analysis and optimization. This paper makes three main contributions: (i) an end-to-end, off-chip congestion-aware and packaging-adaptive analytical framework for detailed analysis, (ii) hardware software co-optimization incorporating diagonal links, on-chip redistribution, and non-uniform workload partitioning to optimize the framework, and (iii) using metaheuristics (genetic algorithms, GA) and mixed integer quadratic programming (MIQP) to solve the optimized framework. Experimental results demonstrate significant performance improvements for CNNs and Vision Transformers, showcasing up to 1.58x and 2.7x EdP (Energy delay Product) improvement using GA and MIQP, respectively.

MCMComm: Hardware-Software Co-Optimization for End-to-End Communication in Multi-Chip-Modules

TL;DR

MCMComm tackles the end-to-end communication bottlenecks in multi-chip-module accelerators by introducing a congestion-aware, packaging-adaptive analytical framework. It couples end-to-end latency and energy-delay product modeling with hardware-software co-optimization, including diagonal links, on-package redistribution, and fine-grained pipelining. Scheduling is solved via two approaches—Genetic Algorithm (GA) and Mixed Integer Quadratic Programming (MIQP)—demonstrating up to 1.58x EdP improvement with GA and 2.7x with MIQP for CNNs and Vision Transformers, respectively. The work provides a practical pathway to scalable, energy-efficient MCM-based AI accelerators, accounting for memory type, packaging, and interconnect characteristics across diverse workloads.

Abstract

Increasing AI computing demands and slowing transistor scaling have led to the advent of Multi-Chip-Module (MCMs) based accelerators. MCMs enable cost-effective scalability, higher yield, and modular reuse by partitioning large chips into smaller chiplets. However, MCMs come at an increased communication cost, which requires critical analysis and optimization. This paper makes three main contributions: (i) an end-to-end, off-chip congestion-aware and packaging-adaptive analytical framework for detailed analysis, (ii) hardware software co-optimization incorporating diagonal links, on-chip redistribution, and non-uniform workload partitioning to optimize the framework, and (iii) using metaheuristics (genetic algorithms, GA) and mixed integer quadratic programming (MIQP) to solve the optimized framework. Experimental results demonstrate significant performance improvements for CNNs and Vision Transformers, showcasing up to 1.58x and 2.7x EdP (Energy delay Product) improvement using GA and MIQP, respectively.
Paper Structure (44 sections, 18 equations, 13 figures, 3 tables, 1 algorithm)

This paper contains 44 sections, 18 equations, 13 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Multi-Chip-Modules
  4. Scheduling in Multi-Core/MCM Accelerators
  5. Motivation
  6. Layer-by-Layer Workload Partition Optimization
  7. DRAM/HBM need different analytical models
  8. Packaging Needs Tailored Optimization
  9. Inefficient Software-Only Optimization
  10. Heuristics-Based Suboptimal Solution
  11. End-to-end Analytical Modeling
  12. Types of MCM Systems
  13. Scheduling Space Formulation
  14. Hardware Configurations
  15. Machine Learning Task
  16. ...and 29 more sections

Figures (13)

  • Figure 1: MCMComm system with NPU-based chiplets, challenges, and key ideas, optimized LS scheduling space, and real-time applications.
  • Figure 2: MCMComm framework with genetic algorithm and non-integer programming schedulers showing different input knobs. The framework is packaging-adaptive as shown by four types of chiplets showing different positions of main memory (DRAM/HBM) in 2.5D and 3D packaging. The framework separately models high-BW and low-BW off-chip cases making it congestion-aware.
  • Figure 3: Modeling results when all chiplets are pulling 1 GB message from the memory over a 4$\times$4 Mesh. Node 16 denotes the memory node. DRAM/HBM bandwidth is 60 GB/s and 1,024 GB/s, and Low/High NoP link bandwidth is 60 GB/s and 120 GB/s, respectively. (a)--(c) Network utilization heat map with different memory types and placements, when NoP bandwidth is 60 GB/s. (d) Total network communication latencies.
  • Figure 4: Illustration of chiplet topology for 4 types of systems. The dashed lines represent chiplet allocation results. Each chiplet has a local index (x, y).
  • Figure 5: Illustration of Congestion during data collection and effects of diagonal links.
  • ...and 8 more figures