Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Family of Distributions of Random Subsets for Controlling Positive and Negative Dependence

Takahiro Kawashima, Hideitsu Hino

TL;DR

The paper introduces discrete kernel point processes (DKPPs) as a flexible family of distributions over random subsets on the powerset $2^\mathcal{Y}$, parameterized by a kernel matrix $\bm{L}$ and a scalar function $\phi$ to control positive and negative dependence. DKPPs encompass DPPs (via $\phi=\log$) and Boltzmann machines (via quadratic $\phi$), enabling smooth transitions between repulsive and attractive interactions through a Box–Cox style transformation $\phi_{\beta,\lambda}$. It develops practical inference and learning tools, including mean-field approximations with Rao–Blackwellization for normalizing constants, ratio matching for kernel learning, and sampling strategies (MCMC/Langevin), alongside efficient marginal/conditional probability computations. Empirical results demonstrate controllable dependence, effective subset acquisition, and advantageous learning behavior on datasets like MNIST and Amazon Baby Registry, highlighting the method’s applicability and scalability. Overall, DKPPs offer a principled and pragmatic pathway to bridging well-known discrete point processes and probabilistic models with tractable computation for real-world tasks requiring diverse subset selections.

Abstract

Positive and negative dependence are fundamental concepts that characterize the attractive and repulsive behavior of random subsets. Although some probabilistic models are known to exhibit positive or negative dependence, it is challenging to seamlessly bridge them with a practicable probabilistic model. In this study, we introduce a new family of distributions, named the discrete kernel point process (DKPP), which includes determinantal point processes and parts of Boltzmann machines. We also develop some computational methods for probabilistic operations and inference with DKPPs, such as calculating marginal and conditional probabilities and learning the parameters. Our numerical experiments demonstrate the controllability of positive and negative dependence and the effectiveness of the computational methods for DKPPs.

A Family of Distributions of Random Subsets for Controlling Positive and Negative Dependence

TL;DR

The paper introduces discrete kernel point processes (DKPPs) as a flexible family of distributions over random subsets on the powerset , parameterized by a kernel matrix and a scalar function to control positive and negative dependence. DKPPs encompass DPPs (via ) and Boltzmann machines (via quadratic ), enabling smooth transitions between repulsive and attractive interactions through a Box–Cox style transformation . It develops practical inference and learning tools, including mean-field approximations with Rao–Blackwellization for normalizing constants, ratio matching for kernel learning, and sampling strategies (MCMC/Langevin), alongside efficient marginal/conditional probability computations. Empirical results demonstrate controllable dependence, effective subset acquisition, and advantageous learning behavior on datasets like MNIST and Amazon Baby Registry, highlighting the method’s applicability and scalability. Overall, DKPPs offer a principled and pragmatic pathway to bridging well-known discrete point processes and probabilistic models with tractable computation for real-world tasks requiring diverse subset selections.

Abstract

Positive and negative dependence are fundamental concepts that characterize the attractive and repulsive behavior of random subsets. Although some probabilistic models are known to exhibit positive or negative dependence, it is challenging to seamlessly bridge them with a practicable probabilistic model. In this study, we introduce a new family of distributions, named the discrete kernel point process (DKPP), which includes determinantal point processes and parts of Boltzmann machines. We also develop some computational methods for probabilistic operations and inference with DKPPs, such as calculating marginal and conditional probabilities and learning the parameters. Our numerical experiments demonstrate the controllability of positive and negative dependence and the effectiveness of the computational methods for DKPPs.
Paper Structure (29 sections, 8 theorems, 39 equations, 16 figures, 1 algorithm)

This paper contains 29 sections, 8 theorems, 39 equations, 16 figures, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Related Works
  3. PRELIMINARY
  4. Supermodular and Submodular Functions
  5. Positive and Negative Dependence
  6. Operator Monotonicity and Convexity
  7. DISCRETE KERNEL POINT PROCESSES
  8. Positive and Negative Dependence of DKPPs
  9. OPERATIONS AND INFERENCE OF DKPPs
  10. Mode Exploration
  11. Sampling
  12. Normalizing Constant and Expectation
  13. Marginal and Conditional Probabilities
  14. LEARNING DKPPs
  15. EXPERIMENTS
  16. ...and 14 more sections

Key Result

Theorem 2.4

friedland2013 Suppose that $\phi$ is a real continuous function on the interval $\mathcal{E} \subset \mathbb{R}$ and that $\phi$ is a primitive of the operator monotone function $\phi'$ on $\mathcal{E}$. Then, for every $N \times N$ Hermitian matrix $\bm{X}$ whose eigenvalues are all in $\mathcal{E} is supermodular. If $\phi$ is a primitive of the operator antitone $\phi'$, the set function $f$ is

Figures (16)

  • Figure 1: SCATTERED
  • Figure 2: GATHERED
  • Figure 4: Number of distinct classes within the acquired subsets.
  • Figure 5: Evaluated gaps $\log Z^{\mathrm{approx}}_\phi(\bm{L}) - \log Z_\phi(\bm{L})$.
  • Figure 8: Learning curves with media. The LPSD takes $7.925 E-2s$ per iteration and ratio matching takes $2.174 E-3s$ per iteration.
  • ...and 11 more figures

Theorems & Definitions (17)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Theorem 2.4
  • Definition 3.1
  • Proposition 3.2
  • proof
  • Proposition 3.3
  • Corollary 3.4
  • Proposition 3.5
  • ...and 7 more