Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

S2DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generation

Lin Zhao, Yushu Wu, Aleksei Lebedev, Dishani Lahiri, Meng Dong, Arpit Sahni, Michael Vasilkovsky, Hao Chen, Ju Hu, Aliaksandr Siarohin, Sergey Tulyakov, Yanzhi Wang, Anil Kag, Yanyu Li

TL;DR

Problem: On-device streaming video generation with high fidelity is challenging due to the quadratic attention cost in diffusion transformers. Approach: S2DiT introduces a sandwich diffusion transformer that interleaves LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA) with a budget-aware dynamic programming search, and employs offline cached knowledge distillation followed by distribution-matching distillation and self-forcing to enable few-step autoregressive streaming. Contributions: a mobile-friendly backbone with DP-based architecture search, a two-stage distillation pipeline that transfers large-teacher fidelity to a compact model, and streaming-capable auto-regressive generation achieving on-device operation at over 10 FPS. Significance: demonstrates practical, high-fidelity, low-latency streaming diffusion on mobile devices, narrowing the gap between server-grade models and on-device capabilities.

Abstract

Diffusion Transformers (DiTs) have recently improved video generation quality. However, their heavy computational cost makes real-time or on-device generation infeasible. In this work, we introduce S2DiT, a Streaming Sandwich Diffusion Transformer designed for efficient, high-fidelity, and streaming video generation on mobile hardware. S2DiT generates more tokens but maintains efficiency with novel efficient attentions: a mixture of LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA). Based on this, we uncover the sandwich design via a budget-aware dynamic programming search, achieving superior quality and efficiency. We further propose a 2-in-1 distillation framework that transfers the capacity of large teacher models (e.g., Wan 2.2-14B) to the compact few-step sandwich model. Together, S2DiT achieves quality on par with state-of-the-art server video models, while streaming at over 10 FPS on an iPhone.

S2DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generation

TL;DR

Problem: On-device streaming video generation with high fidelity is challenging due to the quadratic attention cost in diffusion transformers. Approach: S2DiT introduces a sandwich diffusion transformer that interleaves LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA) with a budget-aware dynamic programming search, and employs offline cached knowledge distillation followed by distribution-matching distillation and self-forcing to enable few-step autoregressive streaming. Contributions: a mobile-friendly backbone with DP-based architecture search, a two-stage distillation pipeline that transfers large-teacher fidelity to a compact model, and streaming-capable auto-regressive generation achieving on-device operation at over 10 FPS. Significance: demonstrates practical, high-fidelity, low-latency streaming diffusion on mobile devices, narrowing the gap between server-grade models and on-device capabilities.

Abstract

Diffusion Transformers (DiTs) have recently improved video generation quality. However, their heavy computational cost makes real-time or on-device generation infeasible. In this work, we introduce S2DiT, a Streaming Sandwich Diffusion Transformer designed for efficient, high-fidelity, and streaming video generation on mobile hardware. S2DiT generates more tokens but maintains efficiency with novel efficient attentions: a mixture of LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA). Based on this, we uncover the sandwich design via a budget-aware dynamic programming search, achieving superior quality and efficiency. We further propose a 2-in-1 distillation framework that transfers the capacity of large teacher models (e.g., Wan 2.2-14B) to the compact few-step sandwich model. Together, S2DiT achieves quality on par with state-of-the-art server video models, while streaming at over 10 FPS on an iPhone.
Paper Structure (30 sections, 15 equations, 6 figures, 8 tables, 1 algorithm)

This paper contains 30 sections, 15 equations, 6 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Design Overview
  5. Efficient Attention
  6. LinConv Hybrid Attention (LCHA)
  7. Stride Self-Attention (SSA)
  8. Sandwich Diffusion Transformer
  9. Budget-aware Block Allocation.
  10. Attention Search Algorithm.
  11. 2-in-1 Distillation
  12. Experiments
  13. Setup
  14. Qualitative Results
  15. Quantitative Results
  16. ...and 15 more sections

Figures (6)

  • Figure 1: Illustration of the framework for obtaining S2DiT. LCHA integrates a linear attention path with a local convolution path at high resolution, while SSA compresses the spatial representation for efficient global context modeling. The final S2DiT is derived by combining these two efficient attention designs with the attention search algorithm.
  • Figure 2: Visual comparisons. For Wan-1.3B wan2025wan and LTX-2B LTX-video, videos are generated using their official default inference resolutions with the same prompts.
  • Figure A1: Visualization of attention maps from our proposed linear attention.
  • Figure A2: Qualitative comparison on vertical videos.
  • Figure A3: Qualitative comparison on vertical videos.
  • ...and 1 more figures