InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling

TL;DR

InstGenIE addresses the bottleneck of diffusion-based image editing by exploiting mask sparsity to skip unmasked computations and reuse cached activations. It introduces a bubble-free cache-loading pipeline, a continuous batching scheme tailored to diffusion denoising, and a mask-aware load-balancing scheduler, all implemented on Diffusers with host-memory caches. The approach yields up to 3x throughput and up to 14.7x latency reduction while maintaining image quality across diverse models and GPUs, enabling practical, scalable production editing services. This work advances diffusion-model serving by integrating mask-guided computation, dynamic pipeline loading, and intelligent request routing to significantly improve cluster-level efficiency.

Abstract

Generative image editing using diffusion models has become a prevalent application in today's AI cloud services. In production environments, image editing typically involves a mask that specifies the regions of an image template to be edited. The use of masks provides direct control over the editing process and introduces sparsity in the model inference. In this paper, we present InstGenIE, a system that efficiently serves image editing requests. The key insight behind InstGenIE is that image editing only modifies the masked regions of image templates while preserving the original content in the unmasked areas. Driven by this insight, InstGenIE judiciously skips redundant computations associated with the unmasked areas by reusing cached intermediate activations from previous inferences. To mitigate the high cache loading overhead, InstGenIE employs a bubble-free pipeline scheme that overlaps computation with cache loading. Additionally, to reduce queuing latency in online serving while improving the GPU utilization, InstGenIE proposes a novel continuous batching strategy for diffusion model serving, allowing newly arrived requests to join the running batch in just one step of denoising computation, without waiting for the entire batch to complete. As heterogeneous masks induce imbalanced loads, InstGenIE also develops a load balancing strategy that takes into account the loads of both computation and cache loading. Collectively, InstGenIE outperforms state-of-the-art diffusion serving systems for image editing, achieving up to 3x higher throughput and reducing average request latency by up to 14.7x while ensuring image quality.

Figures (16)

• Figure 1: A virtual try-on example of image editing using a SDXL model on H800. InstGenIE achieves a model inference speedup of $1.7\times$ and ensures image quality. The Rightmost image: Naively disregarding unmasked regions in image editing will distort the output image.
• Figure 2: A simplified illustration of diffusion model inference. A darker cells/cuboid means it is masked.
• Figure 3: Mask ratio distributions of our traces (Left) and public trace katz (Right).
• Figure 4: Left: Inference latency of a request using different cache loading methods. Middle: Queuing times undergone by requests with static batching anyscale_continuous_batching and InstGenIE's continuous batching under different requst traffic. Right: P95 tail latency of requests with naive load balance and InstGenIE's mask-aware load balance. RPS: request per second.
• Figure 5: Main computations in a transformer block. A darker cell/cuboid means it contains more information about the masked tokens. We omit LayerNorm layernorm, GeLU, and dropout for simplicity, which will not affect the results.