LabelCoRank: Revolutionizing Long Tail Multi-Label Classification with Co-Occurrence Reranking
Yan Yan, Junyuan Liu, Bo-Wen Zhang
TL;DR
LabelCoRank tackles the long-tail issue in multi-label text classification by introducing a dual-stage label reranking that leverages label co-occurrence and dataset-wide frequency distributions. The approach integrates these enhanced label representations back into the text feature space via a label-aware multi-head attention mechanism, guided by a two-loss objective that balances text and label signals. Across MAG-CS, PubMed, and AAPD, LabelCoRank achieves strong overall performance, with notable gains on tail labels and robust ablation results confirming the contributions of label ranking, positional encoding, and extended label fusion. The work demonstrates that incorporating label relationships dynamically per sample significantly improves tail-label prediction without compromising head-label accuracy, offering practical benefits for large-scale, label-rich NLP tasks.
Abstract
Motivation: Despite recent advancements in semantic representation driven by pre-trained and large-scale language models, addressing long tail challenges in multi-label text classification remains a significant issue. Long tail challenges have persistently posed difficulties in accurately classifying less frequent labels. Current approaches often focus on improving text semantics while neglecting the crucial role of label relationships. Results: This paper introduces LabelCoRank, a novel approach inspired by ranking principles. LabelCoRank leverages label co-occurrence relationships to refine initial label classifications through a dual-stage reranking process. The first stage uses initial classification results to form a preliminary ranking. In the second stage, a label co-occurrence matrix is utilized to rerank the preliminary results, enhancing the accuracy and relevance of the final classifications. By integrating the reranked label representations as additional text features, LabelCoRank effectively mitigates long tail issues in multi-labeltext classification. Experimental evaluations on popular datasets including MAG-CS, PubMed, and AAPD demonstrate the effectiveness and robustness of LabelCoRank.