Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Classifier Weighted Mixture models

Elouan Argouarc'h, François Desbouvries, Eric Barat, Eiji Kawasaki, Thomas Dautremer

TL;DR

Classic finite mixtures are limited by fixed weights; the paper introduces Classifier Weighted Mixtures (CWM) where weights become functions of the input, defined via a classifier, increasing expressivity while keeping the PDF tractable and sampling explicit. The approach derives conditions for a valid PDF and provides a latent-variable interpretation, along with a practical parameterization using invertible mappings (NF-style) and a neural network classifier. Reparameterization-gradient-friendly training is enabled through Rao-Blackwellization, and density-estimation experiments show competitive likelihood with fewer parameters than comparable normalizing-flow models and Gaussian mixtures. Overall, CWMs offer a flexible, gradient-friendly alternative for variational inference and density modeling that maintains computational tractability.

Abstract

This paper proposes an extension of standard mixture stochastic models, by replacing the constant mixture weights with functional weights defined using a classifier. Classifier Weighted Mixtures enable straightforward density evaluation, explicit sampling, and enhanced expressivity in variational estimation problems, without increasing the number of components nor the complexity of the mixture components.

Classifier Weighted Mixture models

TL;DR

Classic finite mixtures are limited by fixed weights; the paper introduces Classifier Weighted Mixtures (CWM) where weights become functions of the input, defined via a classifier, increasing expressivity while keeping the PDF tractable and sampling explicit. The approach derives conditions for a valid PDF and provides a latent-variable interpretation, along with a practical parameterization using invertible mappings (NF-style) and a neural network classifier. Reparameterization-gradient-friendly training is enabled through Rao-Blackwellization, and density-estimation experiments show competitive likelihood with fewer parameters than comparable normalizing-flow models and Gaussian mixtures. Overall, CWMs offer a flexible, gradient-friendly alternative for variational inference and density modeling that maintains computational tractability.

Abstract

This paper proposes an extension of standard mixture stochastic models, by replacing the constant mixture weights with functional weights defined using a classifier. Classifier Weighted Mixtures enable straightforward density evaluation, explicit sampling, and enhanced expressivity in variational estimation problems, without increasing the number of components nor the complexity of the mixture components.
Paper Structure (16 sections, 17 equations, 4 figures, 1 table)

This paper contains 16 sections, 17 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. From standard mixtures to CWM
  3. A classifier based construction
  4. The underlying latent variable construction
  5. Change of base distribution and practical construction
  6. Application to variational estimation problems
  7. Parameterization
  8. Base distribution
  9. Classifying function
  10. Invertible mapping
  11. A resulting model
  12. Application to Variational estimation problems
  13. Density Estimation
  14. Reparameterization Gradients
  15. Experiments
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Classical mixtures (left) can be extended to mixtures with functional and non-necessarily constant weights (right).
  • Figure 2: Classifier $\alpha_k(u)$ and corresponding weights $\pi_k(x)$.
  • Figure 3: Directed graph associated with the CWM construction
  • Figure 4: Image - samples - CWM - RNVP - NSF - GMM