HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction
Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, Yong Li
TL;DR
This work redefines citation prediction by introducing core citations, the most foundational references, and presents HLM-Cite, a two-stage hybrid workflow that combines curriculum-finetuned embeddings for scalable retrieval with an LLM-based agentic ranking system for deep textual reasoning. The retrieval stage forms a candidate set and the ranking stage, guided by a Guider, Analyzer, and Decider, ranks candidates via one-shot reasoning to identify the core citations. Evaluated on MAG-derived cross-field data with up to 100K candidates, HLM-Cite achieves up to a 17.6% improvement over state-of-the-art baselines, demonstrating strong scalability and cross-domain applicability. The work highlights practical benefits for researchers and broader implications for leveraging hybrid embeddings and LLMs in long-context scientific NLP tasks, while noting limitations related to LLM reliability and computational demands.
Abstract
Citation networks are critical in modern science, and predicting which previous papers (candidates) will a new paper (query) cite is a critical problem. However, the roles of a paper's citations vary significantly, ranging from foundational knowledge basis to superficial contexts. Distinguishing these roles requires a deeper understanding of the logical relationships among papers, beyond simple edges in citation networks. The emergence of LLMs with textual reasoning capabilities offers new possibilities for discerning these relationships, but there are two major challenges. First, in practice, a new paper may select its citations from gigantic existing papers, where the texts exceed the context length of LLMs. Second, logical relationships between papers are implicit, and directly prompting an LLM to predict citations may result in surface-level textual similarities rather than the deeper logical reasoning. In this paper, we introduce the novel concept of core citation, which identifies the critical references that go beyond superficial mentions. Thereby, we elevate the citation prediction task from a simple binary classification to distinguishing core citations from both superficial citations and non-citations. To address this, we propose $\textbf{HLM-Cite}$, a $\textbf{H}$ybrid $\textbf{L}$anguage $\textbf{M}$odel workflow for citation prediction, which combines embedding and generative LMs. We design a curriculum finetune procedure to adapt a pretrained text embedding model to coarsely retrieve high-likelihood core citations from vast candidates and then design an LLM agentic workflow to rank the retrieved papers through one-shot reasoning, revealing the implicit relationships among papers. With the pipeline, we can scale the candidate sets to 100K papers. We evaluate HLM-Cite across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods.