Revati: Transparent GPU-Free Time-Warp Emulation for LLM Serving

TL;DR

This work addresses the prohibitive cost and slow iteration cycles of tuning large-language-model serving configurations by introducing Revati, a time-warp emulator that runs unmodified serving code at simulation-like speed. It combines transparent CUDA API interception (Device Emulation Layer) with a barrier-based Timekeeper to advance virtual time across distributed processes, preserving causality while skipping GPU waits. Revati achieves end-to-end prediction accuracy below $<$5\% and delivers 5–17× speedups over real GPU execution on vLLM and SGLang across multiple models and configurations, significantly accelerating deployment experimentation. The approach reduces maintenance overhead compared to traditional discrete-event simulators and enables scalable exploration of deployment strategies without requiring hardware-scale GPUs.

Abstract

Deploying LLMs efficiently requires testing hundreds of serving configurations, but evaluating each one on a GPU cluster takes hours and costs thousands of dollars. Discrete-event simulators are faster and cheaper, but they require re-implementing the serving system's control logic -- a burden that compounds as frameworks evolve. We present Revati, a time-warp emulator that enables performance modeling by directly executing real serving system code at simulation-like speed. The system intercepts CUDA API calls to virtualize device management, allowing serving frameworks to run without physical GPUs. Instead of executing GPU kernels, it performs time jumps -- fast-forwarding virtual time by predicted kernel durations. We propose a coordination protocol that synchronizes these jumps across distributed processes while preserving causality. On vLLM and SGLang, Revati achieves less than 5% prediction error across multiple models and parallelism configurations, while running 5-17x faster than real GPU execution.

Figures (10)

• Figure 1: Discrete-event simulators must re-implement the entire control-flow and scheduling logic of serving systems, as a result, the rapid advancements in LLM inference render these simulators perpetually outdated. Revatientirely eliminates this problem by directly running the serving systems in a emulated environment.
• Figure 2: Time Accelerated Emulation. LLM inference engines try to maximize the GPU computation (red); while keeping the CPU overhead (blue) as minimal as possible. Revati exploits this property by skipping GPU operations in wall clock time while preserving the causal semantics of the application using virtual time semantics. This allows us to obtain an order of magnitude acceleration without sacrificing on fidelity.
• Figure 3: CPU vs GPU execution time. GPU execution time dominates serving latency. The low CPU overhead (5-10%) creates the opportunity for order-of-magnitude speedup via time acceleration.
• Figure 4: Coordinating virtual time across processes. The benchmark runner and inference engine run as separate processes, each containing actors (blue) that request time jumps and observers (pink) that receive time updates. Timekeeper coordinates jump requests and broadcasts consistent offsets.
• Figure 5: Barrier-based time synchronization. When actors request different jump durations, Timekeeper advances to the minimum safe offset. Here, Actor 1 requests a 100ms time jump at $t=0$. Before performing the jump, Timekeeper waits for all actors to publish their safe offsets. When Actor 2 requests a 30ms jump at $t=20$, the barrier resolves and Timekeeper advances time by 30ms. Actor 1 still has 50ms remaining, so it requeues a jump request for the remainder. Actor 1's original 100ms jump is thus processed in two chunks, with total wall clock time of just 50ms.