Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding

Shukai Gong, Yiyang Fu, Fengyuan Ran, Quyu Kong, Feng Zhou

Abstract

We propose TPP-SD, a novel approach that accelerates Transformer temporal point process (TPP) sampling by adapting speculative decoding (SD) techniques from language models. By identifying the structural similarities between thinning algorithms for TPPs and speculative decoding for language models, we develop an efficient sampling framework that leverages a smaller draft model to generate multiple candidate events, which are then verified by the larger target model in parallel. TPP-SD maintains the same output distribution as autoregressive sampling while achieving significant acceleration. Experiments on both synthetic and real datasets demonstrate that our approach produces samples from identical distributions as standard methods, but with 2-6$\times$ speedup. Our ablation studies analyze the impact of hyperparameters such as draft length and draft model size on sampling efficiency. TPP-SD bridges the gap between powerful Transformer TPP models and the practical need for rapid sequence sampling.

TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding

Abstract

We propose TPP-SD, a novel approach that accelerates Transformer temporal point process (TPP) sampling by adapting speculative decoding (SD) techniques from language models. By identifying the structural similarities between thinning algorithms for TPPs and speculative decoding for language models, we develop an efficient sampling framework that leverages a smaller draft model to generate multiple candidate events, which are then verified by the larger target model in parallel. TPP-SD maintains the same output distribution as autoregressive sampling while achieving significant acceleration. Experiments on both synthetic and real datasets demonstrate that our approach produces samples from identical distributions as standard methods, but with 2-6 speedup. Our ablation studies analyze the impact of hyperparameters such as draft length and draft model size on sampling efficiency. TPP-SD bridges the gap between powerful Transformer TPP models and the practical need for rapid sequence sampling.

Paper Structure

This paper contains 62 sections, 2 theorems, 33 equations, 27 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary Knowledge
  3. Temporal Point Processes
  4. Thinning Algorithm for TPPs
  5. Speculative Decoding for LLMs
  6. Related Works
  7. Methodology
  8. Thinning Algorithm v.s. Speculative Decoding
  9. CDF-based Transformer TPPs
  10. Encoder.
  11. Decoder.
  12. Naïve autoregressive sampling.
  13. TPP-SD
  14. Drafting.
  15. Verification.
  16. ...and 47 more sections

Key Result

Theorem 1

For a sample $\hat{\tau}_{i+l} \sim g_T(\tau_{i+l}|\cdot)$, define the acceptance threshold as Accept $\hat{\tau}_{i+l}$ if $\epsilon < \alpha$, where $\epsilon \sim \text{Uniform}(0,1)$. This acceptance-rejection procedure generates samples from the adjusted distribution $g^\prime(\tau_{i+l}|\cdot)$ defined in adjusted_dist_time.

Figures (27)

  • Figure 1: (a) Overall architecture of our proposed CDF-based Transformer TPP. (b) The visualization of our proposed TPP-SD sampling method as elaborated in \ref{['TPP-SD-method']}.
  • Figure 2: Poisson
  • Figure 3: Hawkes
  • Figure 4: Multi-Hawkes
  • Figure 6: The impact of draft length $\gamma$ on sampling quality measured by likelihood discrepancy ($\Delta \mathcal{L}$) and distance ($D_{\text{KS}}$ or $D_{\text{WS}}$), and on sampling speed measured by speedup ratio $S_{\text{AR/SD}}$. We conduct all experiments using five random seeds and report the mean and the error band for each metric.
  • ...and 22 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2: Time Rescaling Theorem Meyer1971papangelou1972integrability