Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families

Russell Tsuchida, Cheng Soon Ong, Dino Sejdinovic

TL;DR

We introduce squared neural Poisson point processes (SNEPPPs), parameterising the intensity λ of a Poisson point process as $λ(x)=α\|Vψ(x)\|_2^2$ where $ψ(x)=σ(Wt(x)+b)$ and the final readout is $V$. This construction yields expressivity while enabling exact, closed-form computation of the integrated intensity Λ in many cases via neural network kernels (NNKs), leading to scalable learning with linear-time complexity in the input dimension and at most quadratic in the feature count. When the first layer is fixed, the negative log-likelihood becomes strictly convex in $M=V^TV$, allowing principled MAP/MLE estimation via projected gradient descent. The framework extends to product spaces, supports regularisation and Bayesian extensions, and encompasses valuable special cases such as log-linear and mixture models. Empirical experiments on synthetic and large-scale wildfire data demonstrate competitive performance and substantial scalability, supported by open-source software.

Abstract

We introduce squared neural Poisson point processes (SNEPPPs) by parameterising the intensity function by the squared norm of a two layer neural network. When the hidden layer is fixed and the second layer has a single neuron, our approach resembles previous uses of squared Gaussian process or kernel methods, but allowing the hidden layer to be learnt allows for additional flexibility. In many cases of interest, the integrated intensity function admits a closed form and can be computed in quadratic time in the number of hidden neurons. We enumerate a far more extensive number of such cases than has previously been discussed. Our approach is more memory and time efficient than naive implementations of squared or exponentiated kernel methods or Gaussian processes. Maximum likelihood and maximum a posteriori estimates in a reparameterisation of the final layer of the intensity function can be obtained by solving a (strongly) convex optimisation problem using projected gradient descent. We demonstrate SNEPPPs on real, and synthetic benchmarks, and provide a software implementation. https://github.com/RussellTsuchida/snefy

Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families

TL;DR

We introduce squared neural Poisson point processes (SNEPPPs), parameterising the intensity λ of a Poisson point process as where and the final readout is . This construction yields expressivity while enabling exact, closed-form computation of the integrated intensity Λ in many cases via neural network kernels (NNKs), leading to scalable learning with linear-time complexity in the input dimension and at most quadratic in the feature count. When the first layer is fixed, the negative log-likelihood becomes strictly convex in , allowing principled MAP/MLE estimation via projected gradient descent. The framework extends to product spaces, supports regularisation and Bayesian extensions, and encompasses valuable special cases such as log-linear and mixture models. Empirical experiments on synthetic and large-scale wildfire data demonstrate competitive performance and substantial scalability, supported by open-source software.

Abstract

We introduce squared neural Poisson point processes (SNEPPPs) by parameterising the intensity function by the squared norm of a two layer neural network. When the hidden layer is fixed and the second layer has a single neuron, our approach resembles previous uses of squared Gaussian process or kernel methods, but allowing the hidden layer to be learnt allows for additional flexibility. In many cases of interest, the integrated intensity function admits a closed form and can be computed in quadratic time in the number of hidden neurons. We enumerate a far more extensive number of such cases than has previously been discussed. Our approach is more memory and time efficient than naive implementations of squared or exponentiated kernel methods or Gaussian processes. Maximum likelihood and maximum a posteriori estimates in a reparameterisation of the final layer of the intensity function can be obtained by solving a (strongly) convex optimisation problem using projected gradient descent. We demonstrate SNEPPPs on real, and synthetic benchmarks, and provide a software implementation. https://github.com/RussellTsuchida/snefy
Paper Structure (63 sections, 4 theorems, 40 equations, 2 figures, 3 tables)

This paper contains 63 sections, 4 theorems, 40 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Intensity Functions
  3. Desiderata for Intensity Functions
  4. Contributions
  5. Background
  6. Poisson Point Processes
  7. Squared Neural Families
  8. Flexible Intensity with a Tractable Integral
  9. Squared Neural Poisson Point Process
  10. Our Parameterisation of Intensity Function $\lambda$
  11. Extension to Product Spaces
  12. Integrated Intensity Function
  13. Regularisation
  14. Bayesian Extensions
  15. Closed Form Integration
  16. ...and 48 more sections

Key Result

Proposition 1

Under eq:intensity1 (and therefore eq:intensity2), the intensity function and integrated intensity function are where $\widetilde{\bm{K}}(\bm{x})$ is the PSD matrix with $ij$th entry $\psi_i(\bm{x})\psi_j(\bm{x})$ and $\bm{K}$ is the PSD matrix with $ij$th entry $\kappa(i,j)$,

Figures (2)

  • Figure 1: A SNEPPP spatial intensity function fit (left) to 200,000 events using NASA wildfire data (right) bushfires. In § \ref{['sec:wildfires']}, we discuss a video of a spatio-temporal fit using 100 million events.
  • Figure 2: (Left) Using a naive kernel method or squared GP requires solving a linear system, incurring $\mathcal{O}(N^3)$ time. In contrast, finite feature methods, such as SNEPPP, scale linearly in the number of datapoints. In this setting, the naive method runs out of memory beyond $5.5\%$ of data usage, but even if we had sufficient memory to perform the required calculation, it would take roughly $88$ days to compute using $100\%$ data. In contrast, SNEPPP takes less than 8 hours. (Right) Training SNEPPPs using more data decreases the test NLL, after appropriately accounting for thinning.

Theorems & Definitions (6)

  • Proposition 1
  • Proposition 2
  • Proposition 2
  • proof
  • Proposition 2
  • proof