Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation

Zhehao Yu, Baoquan Zhang, Bingqi Shan, Xinhao Liu, Dongliang Zhou, Guotao Liang, Guangming Ye, Yunming Ye

TL;DR

A novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window, and revealing that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.

Abstract

Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify tokens independently, overlooking the strong co-occurrence patterns between adjacent visual tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window. To construct such phrase units, we analyze token co-occurrence statistics from the training corpus and group frequently co-occurring tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.

SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation

TL;DR

A novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window, and revealing that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.

Abstract

Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify tokens independently, overlooking the strong co-occurrence patterns between adjacent visual tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window. To construct such phrase units, we analyze token co-occurrence statistics from the training corpus and group frequently co-occurring tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.
Paper Structure (22 sections, 9 equations, 4 figures, 4 tables, 2 algorithms)

This paper contains 22 sections, 9 equations, 4 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Autoregressive Image Generation
  4. Accelerating Autoregressive Image Decoding
  5. Preliminaries
  6. Problem Definition
  7. Speculative Jacobi Decoding (SJD)
  8. Methodology
  9. Motivation Analysis.
  10. The Proposed SJD-PV
  11. Phrase Library Construction
  12. Phrase-Level Verification
  13. Theoretical Justification for Effectiveness
  14. Experiments
  15. Experimental Settings
  16. ...and 7 more sections

Figures (4)

  • Figure 1: (a) Existing Token-Level Verification (e.g., SJD): Tokens are verified individually, often causing rejection of locally ambiguous tokens despite their contextual coherence, which breaks visual semantics.(b) Our Phrase-Level Verification (SJD-PV):Contiguous tokens are grouped against a pre-constructed phrase library and verified jointly as coherent phrases This preserves coherent visual semantics to resolve local ambiguity, improving acceptance and accelerating AR image generation.
  • Figure 2: A toy example of the token phrase's visual semantic. While the isolated patch for Token #6302 (top) presents an ambiguous texture, placing it within its contiguous sequence (bottom) exhibits a distinct zebra stripe structure. This validates that visual semantics are fundamentally defined by the collective behavior of neighboring tokens rather than stored independently.
  • Figure 3: Short-range token co-occurrence statistics. We compute token co-occurrence over consecutive positions in tokenized image sequences, revealing strong local correlations among specific visual tokens.
  • Figure 4: Qualitative experiment. Our method on SJD and GSD shows on significant NFE acceleration while maintaining image quality.