Bayesian Semi-supervised Multi-category Classification under Nonparanormality

TL;DR

This work tackles semi-supervised multi-class classification under a nonparanormal framework by modeling transformed features as Gaussian mixtures after a shared componentwise transformation estimated with B-splines. The Bayesian formulation uses conjugate priors and a data-augmentation–driven Gibbs sampler to infer missing labels and mixture parameters, with model selection guided by a low-density boundary criterion for the number of B-spline basis functions. Across extensive binary and multi-class simulations and real datasets (Breast Cancer, Ionosphere), the proposed Nonparanormal (NPN) method demonstrates strong predictive accuracy and robustness to misspecification, particularly when labeled data are limited. The approach offers a flexible, scalable semi-supervised framework that minimizes strong distributional assumptions while delivering competitive performance against established methods.

Abstract

Semi-supervised learning is a model training method that uses both labeled and unlabeled data. This paper proposes a fully Bayes semi-supervised learning algorithm that can be applied to any multi-category classification problem. We assume the labels are missing at random when using unlabeled data in a semi-supervised setting. Suppose we have $K$ classes in the data. We assume that the observations follow $K$ multivariate normal distributions depending on their true class labels after some common unknown transformation is applied to each component of the observation vector. The function is expanded in a B-splines series, and a prior is added to the coefficients. We consider a normal prior on the coefficients and constrain the values to meet the normality and identifiability constraints requirement. The precision matrices of the Gaussian distributions are given a conjugate Wishart prior, while the means are given the improper uniform prior. The resulting posterior is still conditionally conjugate, and the Gibbs sampler aided by a data-augmentation technique can thus be adopted. An extensive simulation study compares the proposed method with several other available methods. The proposed method is also applied to real datasets on diagnosing breast cancer and classification of signals. We conclude that the proposed method has a better prediction accuracy in various cases.