UB-Mesh: a Hierarchically Localized nD-FullMesh Datacenter Network Architecture

TL;DR

UB-Mesh proposes a hierarchically localized nD-FullMesh datacenter network to address the scaling, bandwidth, cost, and availability challenges of large-scale LLM training. It introduces the UB-Mesh-Pod as a 4D-FullMesh building block, the Unified Bus interconnect, APR routing with fast direct-notification fault recovery, and topology-aware optimization for collective operations and parallelization. Key contributions include a 64+1 high-availability design, significant CapEx/OpEx savings (around $2.04\times$ cost-efficiency), and high reliability (availability ~98.8%) with comparable training performance to traditional Clos networks. The framework enables scalable, cost-effective, and robust AI datacenters, scalable up to 8K–1024 NPUs per Pod and beyond through SuperPods, while maintaining 95%+ linearity across varied LLM workloads.

Abstract

As the Large-scale Language Models (LLMs) continue to scale, the requisite computational power and bandwidth escalate. To address this, we introduce UB-Mesh, a novel AI datacenter network architecture designed to enhance scalability, performance, cost-efficiency and availability. Unlike traditional datacenters that provide symmetrical node-to-node bandwidth, UB-Mesh employs a hierarchically localized nD-FullMesh network topology. This design fully leverages the data locality of LLM training, prioritizing short-range, direct interconnects to minimize data movement distance and reduce switch usage. Although UB-Mesh's nD-FullMesh topology offers several theoretical advantages, its concrete architecture design, physical implementation and networking system optimization present new challenges. For the actual construction of UB-Mesh, we first design the UB-Mesh-Pod architecture, which is based on a 4D-FullMesh topology. UB-Mesh-Pod is implemented via a suite of hardware components that serve as the foundational building blocks, including specifically-designed NPU, CPU, Low-Radix-Switch (LRS), High-Radix-Switch (HRS), NICs and others. These components are interconnected via a novel Unified Bus (UB) technique, which enables flexible IO bandwidth allocation and hardware resource pooling. For networking system optimization, we propose advanced routing mechanism named All-Path-Routing (APR) to efficiently manage data traffic. These optimizations, combined with topology-aware performance enhancements and robust reliability measures like 64+1 backup design, result in 2.04x higher cost-efficiency, 7.2% higher network availability compared to traditional Clos architecture and 95%+ linearity in various LLM training tasks.

Figures (22)

• Figure 1: Comparison of Traditional Clos Datacenter Architecture and UB-Mesh
• Figure 2: Parallelism in LLM Training
• Figure 3: Traditional Datacenter Network Architectures
• Figure 4: UB-Mesh's nD-FullMesh Topology and Possible Physical Mapping
• Figure 5: High-Dimensional Topology and Flexible Bandwidth Allocation