vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models

TL;DR

vLLM-Omni tackles the inefficiency of serving complex any-to-any multimodal models by introducing a stage-graph abstraction and a fully disaggregated execution backend. Each stage (AR LLMs or DiT) runs on an independent engine with per-stage batching and a unified inter-stage connector to route data, enabling flexible resource allocation and cross-stage streaming. Empirical results show substantial gains, with up to 91.4% reduction in Job Completion Time on Qwen-Omni models and a 1.26x speedup for diffusion-based tasks, outperforming baselines across multiple models. The work advances practical deployment of multimodal models at scale by decoupling model components, supporting heterogeneous hardware, and minimizing data-transfer overhead.

Abstract

Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically combining multiple autoregressive LLMs, diffusion transformers, and other specialized components) pose substantial challenges for efficient model serving. Existing serving systems are mainly tailored to a single paradigm, such as autoregressive LLMs for text generation or diffusion transformers for visual generation. They lack support for any-to-any pipelines that involve multiple interconnected model components. As a result, developers must manually handle cross-stage interactions, leading to huge performance degradation. We present vLLM-Omni, a fully disaggregated serving system for any-to-any models. vLLM-Omni features a novel stage abstraction that enables users to decompose complex any-to-any architectures into interconnected stages represented as a graph, and a disaggregated stage execution backend that optimizes resource utilization and throughput across stages. Each stage is independently served by an LLM or diffusion engine with per-stage request batching, flexible GPU allocation, and unified inter-stage connectors for data routing. Experimental results demonstrate that vLLM-Omni reduces job completion time (JCT) by up to 91.4% compared to baseline methods. The code is public available at https://github.com/vllm-project/vllm-omni.

Figures (8)

• Figure 1: Any-to-any multimodal model architecture.
• Figure 2: Architecture for existing any-to-any models. (a) Qwen2.5-Omni Qwen2.5-Omni; (b) GLM-Image GLMImage2026; (c) BAGEL bagel.
• Figure 3: vLLM-Omni architecture.
• Figure 4: An example implementation of Qwen2.5-Omni (i.e., stage graph and workflow for the model). Qwen3-Omni is similar.
• Figure 5: Disaggregated data transfer with unified connector.