Multivariate Species Sampling Models

TL;DR

The paper addresses the rigidity of exchangeability in multi-population data by introducing multivariate species sampling processes (mSSP), a unifying framework for dependent nonparametric priors under partial exchangeability. It develops a complete theory around the dependence structure via the partially exchangeable partition function (pEPPF) and its predictive mechanism (mgCRP), with a principal focus on the regular rmSSP subclass that captures most BNP models used in practice. The authors show that borrowing information across populations is governed entirely by shared ties, tying learning mechanisms to observable tie structures, and they provide constructive representations to build new models. The framework is demonstrated through real-tree data and synthetic experiments in a multi-armed bandit setting, highlighting improved species-discovery performance when information-sharing is appropriate and robustness when it is not, with open-source code for replication.

Abstract

Species sampling processes have long served as the fundamental framework for modeling random discrete distributions and exchangeable sequences. However, data arising from distinct but related sources require a broader notion of probabilistic invariance, making partial exchangeability a natural choice. Countless models for partially exchangeable data, collectively known as dependent nonparametric priors, have been proposed. These include hierarchical, nested and additive processes, widely used in statistics and machine learning. Still, a unifying framework is lacking and key questions about their underlying learning mechanisms remain unanswered. We fill this gap by introducing multivariate species sampling models, a new general class of nonparametric priors that encompasses most existing finite- and infinite-dimensional dependent processes. They are characterized by the induced partially exchangeable partition probability function encoding their multivariate clustering structure. We establish their core distributional properties and analyze their dependence structure, demonstrating that borrowing of information across groups is entirely determined by shared ties. This provides new insights into the underlying learning mechanisms, offering, for instance, a principled rationale for the previously unexplained correlation structure observed in existing models. Beyond providing a cohesive theoretical foundation, our approach serves as a constructive tool for developing new models and opens novel research directions for capturing richer dependence structures beyond the framework of multivariate species sampling processes.

Figures (5)

• Figure 1: Probability of observing a new species at the first draw from population $j$, after $n$ observations from population $k$, i.e., $\mathbb{P}(X_{j,1} \notin \{X_{k,1},\ldots, X_{k,n}\})$, plotted as a function of $n$. Results are shown (left to right) for hierarchical Dirichlet processes teh2006hierarchical, nested Dirichlet processes rodriguez2008nested, and additive Dirichlet processes muller2004method for varying values of the correlation measure $\mathbb{C}\mathrm{or}[P_{j}(A), P_{k}(A)]$.
• Figure 2: Empirical distribution functions of the four groups (left) and empirical tie probabilities (right), computed from the full dataset. In the left panel, species within each group are ordered according to their frequency ranking in Group $2$.
• Figure 3: Tree species data: cumulative number of species discovered as a function of the additional sample size for each rmSSP model and the uniform baseline.
• Figure 4: Probabilities of ties based on the true distributions in the simulated scenario.
• Figure 5: Simulated scenario with low probability of ties across populations: Number of species discovered as a function of the additional sample size in the rmSSPs, the uniform model, and the oracle model.

Theorems & Definitions (26)

• Definition 1
• Definition 2
• Definition 3
• Example 1: Hierarchical processes
• Example 2: Nested processes
• Example 3: Additive processes
• Example 4
• Example 5: Hidden hierarchical DP
• Example 6: Semi-hierarchical DP
• Example 7: Processes based on stick-breaking constructions