Semi-Supervised Learning under General Causal Models

TL;DR

The paper tackles semi-supervised learning under general causal models, addressing when unlabeled data can aid learning by examining the independent causal mechanisms principle. It introduces a causal SSL framework that factorises $P(X,Y)=\prod P(v_i|\textbf{Pa}_{v_i})$, learns SCM-based generators, and synthesises labelled data $D_G$ to augment training; it also presents two modelling schemes, CGAN-SSL (disjoint) and GCGAN-SSL (joint), with MMD-based distribution matching and Gumbel-Softmax for joint optimisation. A key contribution is the analysis of Markov Blanket topologies to determine SSL utility and the demonstration of improved predictive performance on seven synthetic graphs and two real datasets, with the joint approach often delivering stronger gains. The findings suggest that explicitly modelling causal mechanisms can unlock the value of unlabelled data in SSL, offering a principled path toward robust learning in real-world causal settings. Overall, the work provides a practical, causally informed SSL framework with data-generation, training procedures, and empirical validation across synthetic and real domains.

Abstract

Semi-supervised learning (SSL) aims to train a machine learning model using both labelled and unlabelled data. While the unlabelled data have been used in various ways to improve the prediction accuracy, the reason why unlabelled data could help is not fully understood. One interesting and promising direction is to understand SSL from a causal perspective. In light of the independent causal mechanisms principle, the unlabelled data can be helpful when the label causes the features but not vice versa. However, the causal relations between the features and labels can be complex in real world applications. In this paper, we propose a SSL framework that works with general causal models in which the variables have flexible causal relations. More specifically, we explore the causal graph structures and design corresponding causal generative models which can be learned with the help of unlabelled data. The learned causal generative model can generate synthetic labelled data for training a more accurate predictive model. We verify the effectiveness of our proposed method by empirical studies on both simulated and real data.

Figures (8)

• Figure 1: Markov Blanket $X_\textbf{MB}=\{X_1,X_5,X_4,X_2\}$
• Figure 2: CG6: Markov Blanket containing $X_S,X_C,X_E$
• Figure 3: Markov Blanket Subgraphs of CG6
• Figure 4: Disjoint approach: factors and scenarios for CG1-CG7
• Figure 5: Curved decision boundary between classes for CG2