PICNIC: Silicon Photonic Interconnected Chiplets with Computational Network and In-memory Computing for LLM Inference Acceleration

TL;DR

The paper tackles the data-movement bottleneck in large language model inference by introducing PICNIC, a 3D-stacked chiplet accelerator that combines non-volatile IMC processing elements with an Inter-PE Computational Network (IPCN) and silicon photonics for inter-chiplet communication. It presents a comprehensive hardware architecture (PEs, IPCN, SCU), a 3D-stacked IC design with an optical interconnect layer, and a CCPG scheme to achieve sub-linear power scaling. An end-to-end LLM orchestration framework (partitioning, mapping, scheduling) is developed to optimize dataflow and resource utilization, validated through hardware-software co-design and cycle-accurate simulations. results show a 3.95× speedup and 30× efficiency improvement over Nvidia A100 before CCPG, and up to 57× efficiency improvement over H100 at similar throughput with CCPG, illustrating PICNIC’s potential to scale energy-efficient LLM inference for larger models.

Abstract

This paper presents a 3D-stacked chiplets based large language model (LLM) inference accelerator, consisting of non-volatile in-memory-computing processing elements (PEs) and Inter-PE Computational Network (IPCN), interconnected via silicon photonic to effectively address the communication bottlenecks. A LLM mapping scheme was developed to optimize hardware scheduling and workload mapping. Simulation results show it achieves $3.95\times$ speedup and $30\times$ efficiency improvement over the Nvidia A100 before chiplet clustering and power gating scheme (CCPG). Additionally, the system achieves further scalability and efficiency improvement with the implementation of CCPG to accommodate larger models, attaining $57\times$ efficiency improvement over Nvidia H100 at similar throughput.

Figures (10)

• Figure 1: LLM Model Size and US Data Center Total Energy Consumption
• Figure 2: Challenges and Solutions for LLM Inference
• Figure 3: PICNIC Hardware Architecture: IPCN, Computing Macros, Interconnects and Instruction Set
• Figure 4: Softmax Compute Unit
• Figure 5: Illustration of Chiplet Clustering with Power Gating Scheme