Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OnePiece: A Large-Scale Distributed Inference System with RDMA for Complex AI-Generated Content (AIGC) Workflows

June Chen, Neal Xu, Gragas Huang, Bok Zhou, Stephen Liu

TL;DR

OnePiece tackles the inefficiencies of multi-stage AIGC pipelines by disaggregating workloads into microservices and deploying a one-sided RDMA network, coupled with a novel double-ring buffer and a dynamic Node Manager for elastic resource allocation. The system partitions pipelines into regionally autonomous Workflow Sets with proxies, workflow instances, and memory-centric databases, enabling high GPU utilization and low inter-node latency without CPU involvement in data movement. Key innovations include a deadlock-free ring buffer for dynamic message sizes, two-mode task scheduling (Individual and Collaboration), and NM-driven dynamic rescheduling and instance sharing, which yield substantial throughput and resource savings in WAN2.1-style workloads. Overall, OnePiece presents a scalable, fault-tolerant production-ready platform for complex AIGC workflows with significant reductions in GPU resource usage and improved throughput under concurrency.

Abstract

The rapid growth of AI-generated content (AIGC) has enabled high-quality creative production across diverse domains, yet existing systems face critical inefficiencies in throughput, resource utilization, and scalability under concurrent workloads. This paper introduces OnePiece, a large-scale distributed inference system with RDMA optimized for multi-stage AIGC workflows. By decomposing pipelines into fine-grained microservices and leveraging one-sided RDMA communication, OnePiece significantly reduces inter-node latency and CPU overhead while improving GPU utilization. The system incorporates a novel double-ring buffer design to resolve deadlocks in RDMA-aware memory access without CPU involvement. Additionally, a dynamic Node Manager allocates resources elastically across workflow stages in response to real-time load. Experimental results demonstrate that OnePiece reduces GPU resource consumption by 16x in Wan2.1 image-to-video generation compared to monolithic inference pipelines, offering a scalable, fault-tolerant, and efficient solution for production AIGC environments.

OnePiece: A Large-Scale Distributed Inference System with RDMA for Complex AI-Generated Content (AIGC) Workflows

TL;DR

OnePiece tackles the inefficiencies of multi-stage AIGC pipelines by disaggregating workloads into microservices and deploying a one-sided RDMA network, coupled with a novel double-ring buffer and a dynamic Node Manager for elastic resource allocation. The system partitions pipelines into regionally autonomous Workflow Sets with proxies, workflow instances, and memory-centric databases, enabling high GPU utilization and low inter-node latency without CPU involvement in data movement. Key innovations include a deadlock-free ring buffer for dynamic message sizes, two-mode task scheduling (Individual and Collaboration), and NM-driven dynamic rescheduling and instance sharing, which yield substantial throughput and resource savings in WAN2.1-style workloads. Overall, OnePiece presents a scalable, fault-tolerant production-ready platform for complex AIGC workflows with significant reductions in GPU resource usage and improved throughput under concurrency.

Abstract

The rapid growth of AI-generated content (AIGC) has enabled high-quality creative production across diverse domains, yet existing systems face critical inefficiencies in throughput, resource utilization, and scalability under concurrent workloads. This paper introduces OnePiece, a large-scale distributed inference system with RDMA optimized for multi-stage AIGC workflows. By decomposing pipelines into fine-grained microservices and leveraging one-sided RDMA communication, OnePiece significantly reduces inter-node latency and CPU overhead while improving GPU utilization. The system incorporates a novel double-ring buffer design to resolve deadlocks in RDMA-aware memory access without CPU involvement. Additionally, a dynamic Node Manager allocates resources elastically across workflow stages in response to real-time load. Experimental results demonstrate that OnePiece reduces GPU resource consumption by 16x in Wan2.1 image-to-video generation compared to monolithic inference pipelines, offering a scalable, fault-tolerant, and efficient solution for production AIGC environments.
Paper Structure (28 sections, 2 theorems, 2 equations, 11 figures)

This paper contains 28 sections, 2 theorems, 2 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Background
  3. RDMA (Remote Direct Memory Access)
  4. NCCL (NVIDIA Collective Communications Library)
  5. Difference between RDMA and NCCL
  6. The AIGC Workflow
  7. System Design Overview
  8. Workflow Set (WS)
  9. Proxy
  10. Workflow Instances
  11. Databases
  12. Workflow Instances
  13. Workflow Message
  14. TaskManager
  15. RequestScheduler ($RS$)
  16. ...and 13 more sections

Key Result

Theorem 1

Consider two stages $X$ and $Y$ with execution times $T_X$ and $T_Y$ respectively, where $T_X < T_Y$. If stage $X$ processes $K$ requests in parallel and stage $Y$ processes $M$ requests in parallel, where $M = \lceil T_Y/T_X * K\rceil$, then the output rates of stages $X$ and $Y$ will be equal.

Figures (11)

  • Figure 1: The system architecture overview of OnePiece
  • Figure 2: The Workflow Set in OnePiece
  • Figure 3: Workflow Message that contains a header and a value produced by the clients or the instances.
  • Figure 4: Two modes of request assignments of RequestScheduler ($RS$).
  • Figure 5: The pipeline example for one instance for stages $X$ and 3 instances for $Y$ with the execution cost $T_X=4s$ and $T_Y=12s$. Stage $X$ is applied with individual mode with 1 worker while stage $Y$ is applied with share mode. Stage $X$ accepts the requests and the stage $Y$ outputs the results every 4 seconds.
  • ...and 6 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2