Deep Sparse Representation-based Classification

TL;DR

The paper addresses the limitations of linear sparse representations by introducing a deep, transductive extension of SRC called DSRC. It integrates a convolutional autoencoder, a sparsity-enforcing sparse coding layer, and a decoder to learn a nonlinear mapping that supports sparse representation, trained end-to-end on both training and test data. Extensive experiments on USPS, SVHN, and UMDAA-01 show that DSRC achieves higher classification accuracy than classical SRC, kernel SRC, autoencoder-SRC, and even some pre-trained network baselines, with notable gains on more challenging datasets. The work demonstrates the effectiveness of learning representations specifically tailored for sparse coding in a transductive setting and provides code for replication.

Abstract

We present a transductive deep learning-based formulation for the sparse representation-based classification (SRC) method. The proposed network consists of a convolutional autoencoder along with a fully-connected layer. The role of the autoencoder network is to learn robust deep features for classification. On the other hand, the fully-connected layer, which is placed in between the encoder and the decoder networks, is responsible for finding the sparse representation. The estimated sparse codes are then used for classification. Various experiments on three different datasets show that the proposed network leads to sparse representations that give better classification results than state-of-the-art SRC methods. The source code is available at: github.com/mahdiabavisani/DSRC.

Figures (5)

• Figure 1: An overview of the proposed deep SRC network. The trainable parameters of sparse coding layer are depicted with solid blue lines. Note that $\mathbf{Z}_{train} = \mathbf{\hat{Z}}_{train}$, and $\mathbf{Z}_{test} \approx \mathbf{\hat{Z}}_{test}= \mathbf{Z}_{train}\mathbf{A}$.
• Figure 2: Sample images from (a) USPS hull1994database, (b) SVHN netzer2011reading, and (c) UMDAA-01 zhang2015domain.
• Figure 3: Visualization of the sparse coding matrix ($\mathbf{A}$) in the experiment with the USPS dataset. Note that for better visualization the absolute value of the transposed $\mathbf{A}$ (i.e. $|\mathbf{A}^T|$) is shown.
• Figure 4: Effect of the number of training samples on the performance of different classification networks. The figure shows five-fold averaged classification accuracies of the methods trained on varying number of training samples in the UMDAA-01 dataset.
• Figure 5: Values of the DSRC's loss function in the experiment on the UMDAA-01 dataset vs iterations. The loss values are scaled to have the maximum value of one.