Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Multivariate Models with Parametric Conditionals

Dmitrij Schlesinger, Boris Flach, Alexander Shekhovtsov

TL;DR

The paper introduces deep multivariate models that represent joint distributions through conditional blocks $p_{\theta_i}(x_i|x_{-i})$ and define the joint as the limiting distribution of an MCMC process. Learning optimises the data likelihood of this limit via a tractable lower bound $L_B(\theta,q,n)$, enabling flexible semi-supervised learning across tasks. It analyzes consistency through detailed balance, presents a general EM-like learning algorithm, and validates the approach on MNIST, Fashion MNIST, and CelebA, demonstrating multiple inference capabilities beyond single-task objectives. The framework offers a task-agnostic, flexible approach to deep multivariate modelling, with the chain length $n$ providing a controllable trade-off between likelihood and consistency and enabling scalable extensions for more complex hierarchies.

Abstract

We consider deep multivariate models for heterogeneous collections of random variables. In the context of computer vision, such collections may e.g. consist of images, segmentations, image attributes, and latent variables. When developing such models, most existing works start from an application task and design the model components and their dependencies to meet the needs of the chosen task. This has the disadvantage of limiting the applicability of the resulting model for other downstream tasks. Here, instead, we propose to represent the joint probability distribution by means of conditional probability distributions for each group of variables conditioned on the rest. Such models can then be used for practically any possible downstream task. Their learning can be approached as training a parametrised Markov chain kernel by maximising the data likelihood of its limiting distribution. This has the additional advantage of allowing a wide range of semi-supervised learning scenarios.

Deep Multivariate Models with Parametric Conditionals

TL;DR

The paper introduces deep multivariate models that represent joint distributions through conditional blocks and define the joint as the limiting distribution of an MCMC process. Learning optimises the data likelihood of this limit via a tractable lower bound , enabling flexible semi-supervised learning across tasks. It analyzes consistency through detailed balance, presents a general EM-like learning algorithm, and validates the approach on MNIST, Fashion MNIST, and CelebA, demonstrating multiple inference capabilities beyond single-task objectives. The framework offers a task-agnostic, flexible approach to deep multivariate modelling, with the chain length providing a controllable trade-off between likelihood and consistency and enabling scalable extensions for more complex hierarchies.

Abstract

We consider deep multivariate models for heterogeneous collections of random variables. In the context of computer vision, such collections may e.g. consist of images, segmentations, image attributes, and latent variables. When developing such models, most existing works start from an application task and design the model components and their dependencies to meet the needs of the chosen task. This has the disadvantage of limiting the applicability of the resulting model for other downstream tasks. Here, instead, we propose to represent the joint probability distribution by means of conditional probability distributions for each group of variables conditioned on the rest. Such models can then be used for practically any possible downstream task. Their learning can be approached as training a parametrised Markov chain kernel by maximising the data likelihood of its limiting distribution. This has the additional advantage of allowing a wide range of semi-supervised learning scenarios.
Paper Structure (24 sections, 7 theorems, 40 equations, 8 figures, 1 table, 1 algorithm)

This paper contains 24 sections, 7 theorems, 40 equations, 8 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learning Limiting Distributions
  4. Multivariate Models with Parametric Conditionals
  5. Practical Learning Algorithm
  6. Experiments
  7. Validation: Learning Consistency
  8. Validation: Chain Length
  9. MNIST / Fashion MNIST
  10. CelebA
  11. Comparison to Other Methods
  12. Conclusion
  13. Proofs and Examples
  14. Proofs for the Generic Algorithm
  15. Further used facts
  16. ...and 9 more sections

Key Result

Proposition 3.0

The lower bound eq:chain has an equivalent reformulation $L_B(\theta, q, n) =$ where $p^r_\theta(x' \,\vert\, x) = p_\theta(x \,\vert\, x')p_\theta(x')/p_\theta(x)$ is the transition kernel of the reverse Markov chain.

Figures (8)

  • Figure 1: Artificial experiment: dependency of the matrix asymmetry on the chain length $n$.
  • Figure 2: Conditional generation for MNIST / Fashion MNIST. Columns correspond to classes.
  • Figure 3: Generation for CelebA, conditioned on the presence of glasses attribute (top) and male attribute (bottom). Each display shows pairs of image and segmentation generated with the attribute switched off (first two rows) and switched on (last two rows).
  • Figure 4: Segmentation from incomplete images / in-painting. First row: original images with hidden parts shown as black squares, second row: ground truth segmentations, third row: predicted segmentations, fourth row: in-painting.
  • Figure 5: Image demixing. First two rows: original image pairs, third row: mixed image, last two rows: demixing results.
  • ...and 3 more figures

Theorems & Definitions (18)

  • Proposition 3.0
  • Proposition 3.0
  • Remark 3.1
  • Remark 3.2
  • Definition 4.1
  • Theorem 4.2
  • Remark 4.3
  • Proposition 1.0
  • proof
  • Proposition 1.0
  • ...and 8 more