Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NCCLbpf: Verified, Composable Policy Execution for GPU Collective Communication

Yusheng Zheng

Abstract

NCCL is the de facto standard for collective GPU communication in large-scale distributed training, relying heavily on plugins to customize runtime behavior. However, these plugins execute as unverified native code within NCCL's address space, risking job crashes, silent state corruption, and downtime from restarts during policy updates. Inspired by kernel extensibility models, we introduce NCCLbpf, a verified, high-performance extension framework embedding a userspace eBPF runtime directly into NCCL's existing plugin interfaces, without modifying NCCL itself. NCCLbpf offers load-time static verification to prevent unsafe plugin execution, structured cross-plugin maps enabling composable policies and closed-loop adaptation, and atomic policy hot-reloads eliminating downtime previously required for policy updates. Evaluations on 8x NVIDIA B300 GPUs connected via NVLink demonstrate that NCCLbpf imposes just 80-130 ns overhead per tuner decision (less than 0.03% of collective latency), prevents all tested unsafe plugin behaviors at load-time, and enables a message-size-aware eBPF policy that improves AllReduce throughput by up to 27% over NCCL's default in the 4-128 MiB range.

NCCLbpf: Verified, Composable Policy Execution for GPU Collective Communication

Abstract

NCCL is the de facto standard for collective GPU communication in large-scale distributed training, relying heavily on plugins to customize runtime behavior. However, these plugins execute as unverified native code within NCCL's address space, risking job crashes, silent state corruption, and downtime from restarts during policy updates. Inspired by kernel extensibility models, we introduce NCCLbpf, a verified, high-performance extension framework embedding a userspace eBPF runtime directly into NCCL's existing plugin interfaces, without modifying NCCL itself. NCCLbpf offers load-time static verification to prevent unsafe plugin execution, structured cross-plugin maps enabling composable policies and closed-loop adaptation, and atomic policy hot-reloads eliminating downtime previously required for policy updates. Evaluations on 8x NVIDIA B300 GPUs connected via NVLink demonstrate that NCCLbpf imposes just 80-130 ns overhead per tuner decision (less than 0.03% of collective latency), prevents all tested unsafe plugin behaviors at load-time, and enables a message-size-aware eBPF policy that improves AllReduce throughput by up to 27% over NCCL's default in the 4-128 MiB range.
Paper Structure (34 sections, 2 figures, 2 tables)

This paper contains 34 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. NCCL and its plugin system.
  4. The safety gap.
  5. eBPF as safe extension mechanism.
  6. Design
  7. Design Tensions
  8. T1: Safety vs. overhead.
  9. T2: Structured sharing vs. flexibility.
  10. T3: Availability vs. consistency during updates.
  11. T4: Compatibility vs. capability.
  12. Threat Model
  13. Architecture
  14. Policy programming model.
  15. Implementation
  16. ...and 19 more sections

Figures (2)

  • Figure 1: NCCLbpf architecture. Policy programs are verified via PREVAIL, JIT-compiled by bpftime, and execute inside the NCCL process. Profiler and tuner communicate through shared typed eBPF maps. The entire system loads as standard NCCL plugins.
  • Figure 2: 8-GPU AllReduce on NVLink. The eBPF policy selects Ring/LL128 for 4--32 MiB and Ring/Simple for 64--192 MiB, improving throughput by up to 27% over NCCL's default NVLS algorithm. A deliberately bad policy (1 channel) shows that policies have real control, including destructive power.