Scaling Routers with In-Package Optics and High-Bandwidth Memories

TL;DR

This work tackles the challenge of scaling internet routers beyond current limits by proposing a router-in-a-package that fuses in-package optics with heterogeneous HBMs and chiplet-based processing. The core innovations are the Split-Parallel Switch (SPS), which passively distributes optical fibers across multiple smaller switches to achieve scalability without extensive OE/EO overhead, and the HBM Switch, which uses a Parallel Frame Interleaving (PFI) scheme to pack traffic into frames that fully exploit the ultra-wide HBM interface. The approach yields promising throughput and scaling: per-switch bandwidth of $81.92\ \,Tb/s$ using $B=4$ HBM stacks and a large, frame-based memory architecture that guarantees 100% throughput for admissible traffic, with modest losses under stress tests (typically <0.05%). A comprehensive set of evaluations on backbone workloads, CAIDA traces, and synthetic cross-datacenter AI traffic demonstrates robustness of the coarse load-balancing strategy, while the analysis highlights power and heat as the primary bottlenecks for future scaling. The proposed design envisions a paradigm shift in router architecture, enabling 1–2 orders of magnitude capacity gains per unit area and prompting rethinking of buffer sizing and power delivery in next-generation networks.

Abstract

This paper aims to apply two major scaling transformations from the computing packaging industry to internet routers: the heterogeneous integration of high-bandwidth memories (HBMs) and chiplets, as well as in-package optics. We propose a novel internet router architecture that employs these technologies to achieve a petabit/sec router within a single integrated package. At the top-level, we introduce a novel split-parallel switch architecture that spatially divides (without processing) the incoming fibers and distributes them across smaller independent switches without intermediate OEO conversions or fine-tuned per-packet load-balancing. This passive spatial division enables scaling at the cost of a coarser traffic load balancing. Yet, through extensive evaluations of backbone network traffic, we demonstrate that differences with fine-tuned approaches are small. In addition, we propose a novel HBM-based shared-memory architecture for the implementation of the smaller independent switches, and we introduce a novel parallel frame interleaving algorithm that packs traffic into frames so that HBM banks are accessed at peak HBM data rates in a cyclical interleaving manner. We further discuss why these new technologies represent a paradigm shift in the design of future internet routers. Finally, we emphasize that power consumption may constitute the primary bottleneck to scaling.

Figures (9)

• Figure 1: The SPS architecture.
• Figure 2: Packaging view of SPS on a 2.5D photonics interposer.
• Figure 3: The architecture of each HBM switch.
• Figure 4: Intuition for the HBM switch operation.
• Figure 5: Packet loss rate (or lack thereof) with Abilene and GEANT workloads and original router sizes.

Theorems & Definitions (5)

• Theorem 1: SRAM size
• Theorem 2: Mimicking with frames only
• Theorem 3: 100% throughput for HBM switch
• Theorem 4: 100% throughput for SPS
• Theorem 5: Mimicking with speedup