Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

TL;DR

The paper tackles learning visual classifiers with scarce labels by exploiting unlabeled data through context-conditioned in-painting with an adversarial loss. It introduces CC-GAN and its augmented variant CC-GAN^2, where a generator fills a masked patch conditioned on surrounding context and a discriminator distinguishes real versus completed (and full generated) images, combining supervised and unsupervised objectives for semi-supervised learning. Empirical results on STL-10 and PASCAL VOC 2007 show that CC-GAN variants outperform several baselines, with CC-GAN^2 often achieving the best results, demonstrating effective use of context to regularize powerful discriminators. The approach scales to large architectures and benefits from optional low-resolution conditioning to handle larger holes, offering a practical method for leveraging unlabeled data in image classification.

Abstract

We introduce a simple semi-supervised learning approach for images based on in-painting using an adversarial loss. Images with random patches removed are presented to a generator whose task is to fill in the hole, based on the surrounding pixels. The in-painted images are then presented to a discriminator network that judges if they are real (unaltered training images) or not. This task acts as a regularizer for standard supervised training of the discriminator. Using our approach we are able to directly train large VGG-style networks in a semi-supervised fashion. We evaluate on STL-10 and PASCAL datasets, where our approach obtains performance comparable or superior to existing methods.

Figures (6)

• Figure 1: (a) Context-encoder of pathak2016, configured for object classification task. (b) Semi-supervised learning with GANs (SSL-GAN). (c) Semi-supervised learning with CC-GANs. In (a-c) the blue network indicates the feature representation being learned (encoder network in the context-encoder model and discriminator network in the GAN and CC-GAN models).
• Figure 2: Architecture of our context-conditional generative adversarial network (CC-GAN). conv(64, 4x4, 2x2) denotes a conv layer with 64 channels, 4x4 kernels and stride 2x2. Each convolution layer is followed by a spatial batch normalization and rectified linear layer. Dashed lines indicate optional pathways.
• Figure 3: STL-10 in-painting with CC-GAN training and varying methods of dropping out the image.
• Figure 4: STL-10 in-painting with combined CC-GAN$^2$ training.
• Figure 5: PASCAL in-painting with CC-GAN.