Bayesian Mixtures Models with Repulsive and Attractive Atoms

TL;DR

This work develops a unified Bayesian nonparametric framework for mixture models with interacting atoms, enabling both repulsive and attractive dependencies among cluster locations through normalised random measures derived from marked point processes. By employing Palm calculus and Laplace functionals, the authors derive closed-form expressions for the posterior, marginal, and predictive distributions, without restricting to finite atom sets, and specialize the theory to Poisson, Gibbs, determinantal, and shot-noise Cox processes. They present two MCMC algorithms—one conditional and one marginal—for Bayesian hierarchical mixtures and illustrate the approaches on simulated and real data, including mixtures of $t$ distributions and galaxy velocities. The results highlight the tradeoffs between density estimation and clustering under misspecification and demonstrate the added flexibility of shot-noise Cox and determinantal priors for capturing cluster structure while maintaining interpretable components.

Abstract

The study of almost surely discrete random probability measures is an active line of research in Bayesian nonparametrics. The idea of assuming interaction across the atoms of the random probability measure has recently spurred significant interest in the context of Bayesian mixture models. This allows the definition of priors that encourage well-separated and interpretable clusters. In this work, we provide a unified framework for the construction and the Bayesian analysis of random probability measures with interacting atoms, encompassing both repulsive and attractive behaviours. Specifically, we derive closed-form expressions for the posterior distribution, the marginal and predictive distributions, which were not previously available except for the case of measures with i.i.d. atoms. We show how these quantities are fundamental both for prior elicitation and to develop new posterior simulation algorithms for hierarchical mixture models. Our results are obtained without any assumption on the finite point process that governs the atoms of the random measure. Their proofs rely on analytical tools borrowed from the Palm calculus theory, which might be of independent interest. We specialise our treatment to the classes of Poisson, Gibbs, and determinantal point processes, as well as in the case of shot-noise Cox processes. Finally, we illustrate the performance of different modelling strategies on simulated and real datasets.

Figures (9)

• Figure 1: $\mathsf{P}(K_n = k, \bm Y^* \in \mathrm d \bm y^*)$ when $n=5$, $a=1$, as a function of the parameter $x$ under different settings. Left plot, setting (I) under the Poisson process () and DPP () prior. Middle plot: setting (I) () and setting (II) () under the DPP prior. Right plot setting (I) () and setting (III) () under the DPP prior.
• Figure 2: From left to right: posterior co-clustering matrix under the normalised IFPP (a) and SNCP (b) mixtures, posterior distribution of the number of clusters (c), density estimates (d) for the simulated example in Section \ref{['sec:simu1']}. In the posterior co-clustering matrices, data are sorted from the smallest to the largest.
• Figure 3: Posterior distribution of the number of clusters (a) and density estimates (b) for the simulated example in Section \ref{['sec:simu2']}.
• Figure 4: Analysis of the Shapley galaxy data under the three different models.
• Figure 5: Posterior mean of the number of clusters $K_n$ for the simulation in Appendix \ref{['app:dpp_sensitivity']}. Boxplots refer to 100 independent replicates.

Theorems & Definitions (57)

• Example 1: Determinantal point processes
• Remark 1: Number of points in a DPP
• Remark 2: Projection DPPs
• Remark 3: DPP densities
• Remark 4: Non finite measures on $\mathbb{X}$
• Example 2: Shot-Noise Cox Processes
• Theorem 1
• Example 3: Determinantal point process
• Example 4: Shot-Noise Cox process
• Theorem 2