MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification

TL;DR

MultiMatch addresses semi-supervised text classification by unifying co-training and consistency regularization through a three-head architecture and a three-fold pseudo-label weighting module. By combining head agreement, self-adaptive thresholds, and Average Pseudo-Margins, it filters and weights pseudo-labels to minimize noise and exploit informative examples, achieving state-of-the-art performance on multiple USB NLP benchmarks and robustness under extreme class imbalance. The work includes extensive ablations, an analysis of pseudo-label quality, and a multimodal extension to CrisisMMD, underscoring the method's versatility and practical impact for resource-limited settings. Overall, MultiMatch offers a principled, hybrid SSL framework with strong empirical gains and a concrete pathway for handling imbalanced data in real-world text classification tasks.

Abstract

We introduce MultiMatch, a novel semi-supervised learning (SSL) algorithm combining the paradigms of co-training and consistency regularization with pseudo-labeling. At its core, MultiMatch features a pseudo-label weighting module designed for selecting and filtering pseudo-labels based on head agreement and model confidence, and weighting them according to the perceived classification difficulty. This novel module enhances and unifies three existing techniques -- heads agreement from Multihead Co-training, self-adaptive thresholds from FreeMatch, and Average Pseudo-Margins from MarginMatch -- resulting in a holistic approach that improves robustness and performance in SSL settings. Experimental results on benchmark datasets highlight the superior performance of MultiMatch, i.e., MultiMatch achieves state-of-the-art results on 8 out of 10 setups from 5 natural language processing datasets and ranks first according to the Friedman test among 21 methods. Furthermore, MultiMatch demonstrates exceptional robustness in highly imbalanced settings, outperforming the second-best approach by 3.26%, a critical advantage for real-world text classification tasks. Our code is available on GitHub.

Figures (3)

• Figure 1: MultiMatch architecture with three heads. Orange and red lines show the path of the weakly augmented and strongly augmented samples, respectively. The green PLWM module receives the $q_b^1$ and $q_b^2$ predictions on the weakly augmented samples from heads $H_1$ and $H_2$, then generates the pseudo-labels $\hat{q}_b^3$ and the corresponding weights $\mathcal{W}_{Multi}^3$. They are used together with the prediction $Q_b^3$ of head $H_3$ on the strongly augmented samples for computing the unsupervised loss $\mathcal{L}_u^3$ using the cross-entropy function. While the diagram shows the unlabeled loss computation for head $H_3$, similar computations are used for $H_1$ and $H_2$.
• Figure 2: Mask rate and impurity on AG News with 200 labels.
• Figure 3: Mask rate (left) and impurity (right) on AG News with imbalance factor 100.