Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semi-Supervised Learning with Deep Generative Models

Diederik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling

TL;DR

It is shown that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

Abstract

The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

Semi-Supervised Learning with Deep Generative Models

TL;DR

It is shown that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

Abstract

The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

Paper Structure

This paper contains 16 sections, 10 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Deep Generative Models for Semi-supervised Learning
  3. Scalable Variational Inference
  4. Lower Bound Objective
  5. Latent Feature Discriminative Model Objective
  6. Generative Semi-supervised Model Objective
  7. Optimisation
  8. Computational Complexity
  9. Experimental Results
  10. Benchmark Classification
  11. Results:
  12. Conditional Generation
  13. Image Classification
  14. Optimization details
  15. Discussion and Conclusion
  16. ...and 1 more sections

Figures (2)

  • Figure 1: (a) Visualisation of handwriting styles learned by the model with 2D $\hbox{$\hbox{$\mathbf{z}$}$}$-space. (b,c) Analogical reasoning with generative semi-supervised models using a high-dimensional $\hbox{$\hbox{$\mathbf{z}$}$}$-space. The leftmost columns show images from the test set. The other columns show analogical fantasies of $\hbox{$\hbox{$\mathbf{x}$}$}$ by the generative model, where the latent variable $\hbox{$\hbox{$\mathbf{z}$}$}$ of each row is set to the value inferred from the test-set image on the left by the inference network. Each column corresponds to a class label $\hbox{$\hbox{$\mathbf{y}$}$}$.
  • Figure : Learning in model M1