Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonstationary Sparse Spectral Permanental Process

Zicheng Sun, Yixuan Zhang, Zenan Ling, Xuhui Fan, Feng Zhou

TL;DR

This work tackles the limitations of permanental processes for point-process modeling by enabling flexible nonstationary kernels via sparse spectral representations. It introduces NSSPP and its deep variant DNSSPP, which learn nonstationary kernels directly from data and reduce inference from $O(N^3)$ to $O(NR^2)$ through a low-rank spectral factorization; a Laplace-approximate inference scheme is developed to scale to larger datasets. Empirical results on synthetic and real data show that NSSPP/DNSSPP achieve competitive performance on stationary settings while delivering clear advantages under pronounced nonstationarity, with DNSSPP offering greater expressiveness at the cost of higher computation. The work highlights the importance of nonstationarity in permanental processes and demonstrates a practical, scalable approach to learn flexible kernels within Gaussian Cox process frameworks.

Abstract

Existing permanental processes often impose constraints on kernel types or stationarity, limiting the model's expressiveness. To overcome these limitations, we propose a novel approach utilizing the sparse spectral representation of nonstationary kernels. This technique relaxes the constraints on kernel types and stationarity, allowing for more flexible modeling while reducing computational complexity to the linear level. Additionally, we introduce a deep kernel variant by hierarchically stacking multiple spectral feature mappings, further enhancing the model's expressiveness to capture complex patterns in data. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our approach, particularly in scenarios with pronounced data nonstationarity. Additionally, ablation studies are conducted to provide insights into the impact of various hyperparameters on model performance.

Nonstationary Sparse Spectral Permanental Process

TL;DR

This work tackles the limitations of permanental processes for point-process modeling by enabling flexible nonstationary kernels via sparse spectral representations. It introduces NSSPP and its deep variant DNSSPP, which learn nonstationary kernels directly from data and reduce inference from to through a low-rank spectral factorization; a Laplace-approximate inference scheme is developed to scale to larger datasets. Empirical results on synthetic and real data show that NSSPP/DNSSPP achieve competitive performance on stationary settings while delivering clear advantages under pronounced nonstationarity, with DNSSPP offering greater expressiveness at the cost of higher computation. The work highlights the importance of nonstationarity in permanental processes and demonstrates a practical, scalable approach to learn flexible kernels within Gaussian Cox process frameworks.

Abstract

Existing permanental processes often impose constraints on kernel types or stationarity, limiting the model's expressiveness. To overcome these limitations, we propose a novel approach utilizing the sparse spectral representation of nonstationary kernels. This technique relaxes the constraints on kernel types and stationarity, allowing for more flexible modeling while reducing computational complexity to the linear level. Additionally, we introduce a deep kernel variant by hierarchically stacking multiple spectral feature mappings, further enhancing the model's expressiveness to capture complex patterns in data. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our approach, particularly in scenarios with pronounced data nonstationarity. Additionally, ablation studies are conducted to provide insights into the impact of various hyperparameters on model performance.
Paper Structure (39 sections, 2 theorems, 42 equations, 8 figures, 3 tables, 1 algorithm)

This paper contains 39 sections, 2 theorems, 42 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Efficient Permanental Process
  4. Nonstationary Kernel
  5. Preliminaries
  6. Permanental Process
  7. Sparse Spectral Kernel
  8. Our Model
  9. Nonstationary Sparse Spectral Permanental Process
  10. Deep Nonstationary Sparse Spectral Permanental Process
  11. Inference
  12. Joint Distribution
  13. Laplace Approximation
  14. Hyperparameter and Complexity
  15. Predictive Distribution
  16. ...and 24 more sections

Key Result

Theorem 3.1

bochner1932vorlesungen A stationary kernel function $k(\mathbf{x}_1, \mathbf{x}_2)=k(\mathbf{x}_1-\mathbf{x}_2): \mathbb{R}^D \rightarrow \mathbb{R}$ is bounded, continuous, and positive definite if and only if it can be represented as: where $\mu(\bm{\omega})$ is a bounded non-negative measure associated to the spectral density $p(\bm{\omega}) = \frac{\mu(\bm{\omega})}{\mu(\mathbb{R}^D)}$.

Figures (8)

  • Figure 1: Stationary Data
  • Figure 2: Nonstationary Data
  • Figure 3: Width v.s. depth
  • Figure 4: Epoch v.s. learning rate
  • Figure 6: The fitting results of the intensity functions from LBPP and DNSSPP on the Redwoods and Taxi datasets. Additional results for various baselines on three datasets are provided in \ref{['additional_exp']}.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Theorem 3.1
  • Theorem 4.1