Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The DMA Streaming Framework: Kernel-Level Buffer Orchestration for High-Performance AI Data Paths

Marco Graziano

Abstract

AI transport libraries move bytes efficiently, but they commonly assume that buffers are already correctly allocated, placed, shared, registered, and safe under completion and teardown pressure. This paper presents dmaplane, a Linux kernel module that makes this missing layer explicit as buffer orchestration. dmaplane exposes a stable kernel UAPI via /dev/dmaplane and composes ring-based command channels, DMA buffer lifecycle management, dma-buf export for cross-device sharing, a kernel-space RDMA engine, NUMA-aware allocation and verification, credit-based flow control, low-overhead observability, and GPU memory integration via PCIe BAR pinning. We evaluate orchestration sensitivity with measurements of NUMA cross-node penalties at DRAM scale, completion-safe flow control under sustained RDMA load, and GPU BAR mapping tiers versus cudaMemcpy. We also demonstrate end-to-end disaggregated inference by transferring KV-cache chunks between two machines using RDMA WRITE WITH IMMEDIATE and reconstructing tensor views on the receiver. RDMA measurements use Soft-RoCE; we distinguish measured results from provider-independent properties by construction.

The DMA Streaming Framework: Kernel-Level Buffer Orchestration for High-Performance AI Data Paths

Abstract

AI transport libraries move bytes efficiently, but they commonly assume that buffers are already correctly allocated, placed, shared, registered, and safe under completion and teardown pressure. This paper presents dmaplane, a Linux kernel module that makes this missing layer explicit as buffer orchestration. dmaplane exposes a stable kernel UAPI via /dev/dmaplane and composes ring-based command channels, DMA buffer lifecycle management, dma-buf export for cross-device sharing, a kernel-space RDMA engine, NUMA-aware allocation and verification, credit-based flow control, low-overhead observability, and GPU memory integration via PCIe BAR pinning. We evaluate orchestration sensitivity with measurements of NUMA cross-node penalties at DRAM scale, completion-safe flow control under sustained RDMA load, and GPU BAR mapping tiers versus cudaMemcpy. We also demonstrate end-to-end disaggregated inference by transferring KV-cache chunks between two machines using RDMA WRITE WITH IMMEDIATE and reconstructing tensor views on the receiver. RDMA measurements use Soft-RoCE; we distinguish measured results from provider-independent properties by construction.
Paper Structure (45 sections, 4 figures, 5 tables)

This paper contains 45 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Goals and non goals
  3. Motivating workloads beyond disaggregated inference
  4. Contributions
  5. Paper organization
  6. Background and Related Work
  7. Kernel primitives used by dmaplane
  8. Positioning and closest neighbors
  9. Architecture and Design Invariants
  10. Architecture overview
  11. Locking model
  12. Design invariants
  13. Core Mechanisms
  14. Channels and ring based dispatch
  15. Buffer lifecycle, mmap, and dma-buf export
  16. ...and 30 more sections

Figures (4)

  • Figure 1: dmaplane block diagram.
  • Figure 2: dma-buf export and per importer attachment mapping.
  • Figure 3: RDMA resource hierarchy.
  • Figure 4: Timeline of send CQ credits and receive window credits in a WRITE WITH IMMEDIATE pipeline.