YODA: Yet Another One-step Diffusion-based Video Compressor

TL;DR

YODA tackles perceptual video compression by extending diffusion-based methods to video through temporal conditioning and efficient one-step denoising. Its three-part architecture—Temporal-Aware AutoEncoder for cross-frame latent generation, a Conditional Latent Coder for entropy-aware context modeling, and a Linear Diffusion Transformer for one-step denoising—delivers superior perceptual quality (LPIPS, DISTS, FID, KID) while maintaining competitive bitrate efficiency. A staged training regime stabilizes learning and balances rate, distortion, and realism, and extensive ablations illuminate the impact of temporal awareness, multiscale conditioning, and DiT denoising. The approach sets a new benchmark for diffusion-based video codecs and offers practical potential for perceptual-focused video coding, with code to be released publicly.

Abstract

While one-step diffusion models have recently excelled in perceptual image compression, their application to video remains limited. Prior efforts typically rely on pretrained 2D autoencoders that generate per-frame latent representations independently, thereby neglecting temporal dependencies. We present YODA--Yet Another One-step Diffusion-based Video Compressor--which embeds multiscale features from temporal references for both latent generation and latent coding to better exploit spatial-temporal correlations for more compact representation, and employs a linear Diffusion Transformer (DiT) for efficient one-step denoising. YODA achieves state-of-the-art perceptual performance, consistently outperforming traditional and deep-learning baselines on LPIPS, DISTS, FID, and KID. Source code will be publicly available at https://github.com/NJUVISION/YODA.

Figures (12)

• Figure 1: (a) YODA adopts a trainable temporal-aware autoencoder, a latent codec that models motion implicitly, and a linear DiT-based denoiser; while (b) current approaches use a frozen autoencoder that operates only in the spatial domain, a latent codec that explicitly encodes motion, and a U-Net denoiser. For the diffusion denoiser, LoRA fine-tuning is applied.
• Figure 2: YODA. The current frame $x_t$ is first processed by the Temporal-Aware Encoder (TA Encoder) to produce $l_t$, and then passed through a channel expansion (Channel-expand) block to obtain $f_t$, which increases the channel dimensionality from 32 to 256. These features are subsequently compressed under the guidance of a Hyper Model. In addition to serving as conditions, the decoded features $\hat{f}_t$ are passed through a channel squeezing (Ch-squeeze) module that reduces the channel dimensionality back to 32, yielding $\tilde{l}_t$. The representation $\tilde{l}_t$ is then denoised by a linear DiT module to obtain $\hat{l}_t$, after which the Temporal-Aware Decoder (TA Decoder) reconstructs the image $\hat{x}_t$. An Extractor forms a temporal feedback loop by extracting multiscale cues $\{\hat{c}_i\}_{i=1}^5$ from the previous reconstruction $\hat{x}_{t-1}$ and injecting them back into the main encoder–decoder backbone. Q, AE, and AD stand for quantization, arithmetic encoding, and arithmetic decoding, respectively.
• Figure 3: Temporal-Aware Autoencoder (TA-AE) augments the standard DC‑AE by incorporating multiscale temporal features, $\{\hat{c}_i\}_{i=1}^5$, extracted from the reference frame $\hat{x}_{t-1}$ through the use of Extractor.
• Figure 4: Conditional Latent Coder (CLC). The previous frame’s feature $\hat{f}_{t-1}$ is processed by depth-wise convolution blocks to produce temporal conditions $\hat{F}_{t}$ and $\hat{F}_h$ for the main encoder–decoder and entropy model.
• Figure 5: Low-Delay IPPP Structure used in YODA. Currently, I-Frame and P-Frame share a similar architecture and the same training.