Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning

TL;DR

The paper tackles short text classification by addressing semantic sparsity and limited labeled data with MI-DELIGHT, a framework that fuses multi-source information through word, POS, and entity graphs and leverages a hierarchical, dual-level contrastive learning strategy.By learning progressively abstract features from instance-level to cluster-level contrastive signals and then performing final classification, MI-DELIGHT captures both fine-grained and high-level patterns in an unsupervised-to-supervised training flow.Empirical results on multiple STC datasets show MI-DELIGHT outperforms various baselines and even several large language models in domain-specific contexts, validating the benefit of structured information integration and hierarchical multi-task learning.The work advances practical STC by enabling better utilization of unlabeled data, richer contextual cues, and task-aware learning dynamics, with potential impact on real-world text understanding where labeled data are scarce.

Abstract

Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets.

Figures (1)

• Figure 1: The overall architecture of MI-DELIGHT. We first generate augmented samples for the input texts. Then, the original corpus $\mathcal{D}_{\text{org}} =\left\{d_{i}^{\text{org}}\right\}_{i=1}^{N}$ and the augmented corpus $\mathcal{D}_{\text{aug}} =\left\{d{_{i}^{\text{aug}}}\right\}_{i=1}^{N}$ are used to construct a word graph $\mathcal{G}_w$, a POS graph $\mathcal{G}_p$ and an entity graph $\mathcal{G}_e$, and the text embeddings $\mathbf{Z}$ are obtained via the text representation learning module. Finally, these embeddings are mapped through different projection heads into different hidden spaces to which ICL, CCL, and cross-entropy (CE) are applied in a certain hierarchical order. From ICL to CCL and then to CE, the task complexity keeps increasing, and the features keep more abstract. Here, feature specificity represents the abstraction level of features.