Self-Attention to Operator Learning-based 3D-IC Thermal Simulation

TL;DR

The paper tackles the challenge of fast, accurate 3D IC thermal simulation under high power density by introducing SAU-FNO, a Self-Attention U-Net Fourier Neural Operator that blends global dependency modeling with local high-frequency detail capture. It fuses a U-FNO architecture with a self-attention mechanism and leverages transfer learning to reduce reliance on extensive high-fidelity data. The method achieves state-of-the-art accuracy and substantial speedups (notably ~842× over traditional FEM tools) while maintaining physical realism and mesh-invariance through Fourier-based operator learning. These results enable efficient design-space exploration and robust thermal management for complex 3D integrated circuits.

Abstract

Thermal management in 3D ICs is increasingly challenging due to higher power densities. Traditional PDE-solving-based methods, while accurate, are too slow for iterative design. Machine learning approaches like FNO provide faster alternatives but suffer from high-frequency information loss and high-fidelity data dependency. We introduce Self-Attention U-Net Fourier Neural Operator (SAU-FNO), a novel framework combining self-attention and U-Net with FNO to capture long-range dependencies and model local high-frequency features effectively. Transfer learning is employed to fine-tune low-fidelity data, minimizing the need for extensive high-fidelity datasets and speeding up training. Experiments demonstrate that SAU-FNO achieves state-of-the-art thermal prediction accuracy and provides an 842x speedup over traditional FEM methods, making it an efficient tool for advanced 3D IC thermal simulations.

Figures (5)

• Figure 1: Our SAU-FNO model architecture. $p(x)$ is the input, $P$ and $Q$ are fully connected neural networks, and $T(x)$ is the output. $\mathcal{F}$ denotes the Fourier transform, R is the parameterization in Fourier space, $\mathcal{F}^{-1}$ is the inverse Fourier transform, $W$ is a linear bias term, and $\alpha$ is the activation function. $U$ denotes a U-Net.
• Figure 2: The self-attention block architecture.
• Figure 3: The stacking structures and floorplan of three 3D chips examined in this work.
• Figure 4: Comparisons between our SAU-FNO predictions and Ground-Truth on Case 1 of Chip1.
• Figure 5: Comparisons between our SAU-FNO predictions and Ground-Truth on Case 2 of Chip1.