Understanding Survey Paper Taxonomy about Large Language Models via Graph Representation Learning
Jun Zhuang, Casey Kennington
TL;DR
This work tackles the challenge of keeping up with rapidly proliferating LLM survey papers by automatically mapping them into a structured taxonomy. It constructs three attributed graph representations (text, co-author, and co-category) and uses graph representation learning, especially on the co-category graph, to classify papers, showing superior performance over fine-tuned language models and even human baselines on a small, imbalanced dataset of 144 papers. A key insight is that weak labels generated by a smaller model can improve the performance of larger models when fine-tuning, highlighting a weak-to-strong generalization potential. The study provides a practical framework for organizing survey literature and aiding newcomers, with implications for scalable literature synthesis in fast-evolving AI domains.
Abstract
As new research on Large Language Models (LLMs) continues, it is difficult to keep up with new research and models. To help researchers synthesize the new research many have written survey papers, but even those have become numerous. In this paper, we develop a method to automatically assign survey papers to a taxonomy. We collect the metadata of 144 LLM survey papers and explore three paradigms to classify papers within the taxonomy. Our work indicates that leveraging graph structure information on co-category graphs can significantly outperform the language models in two paradigms; pre-trained language models' fine-tuning and zero-shot/few-shot classifications using LLMs. We find that our model surpasses an average human recognition level and that fine-tuning LLMs using weak labels generated by a smaller model, such as the GCN in this study, can be more effective than using ground-truth labels, revealing the potential of weak-to-strong generalization in the taxonomy classification task.