Graph-based Retrieval Augmented Generation for Dynamic Few-shot Text Classification

TL;DR

The paper tackles dynamic few-shot text classification under evolving label sets by proposing GORAG, a hyperparameter-free framework that uses a weighted keyword-label graph constructed online across rounds. Retrieval is performed via a minimum-cost spanning tree on the graph, yielding adaptive candidate labels without human thresholds, while an online indexing mechanism enriches the graph with new query keywords. Through extensive experiments on WOS, Reuters, and IFS-Rel, GORAG achieves superior accuracy and robustness to label updates, and ablations confirm the critical roles of edge weighting and online indexing. The approach reduces input length and noise for LLMs and offers scalable, context-rich augmentation for real-world dynamic DFSTC tasks.

Abstract

Text classification is a fundamental task in data mining, pivotal to various applications such as tabular understanding and recommendation. Although neural network-based models, such as CNN and BERT, have demonstrated remarkable performance in text classification, their effectiveness heavily relies on abundant labeled training data. This dependency makes these models less effective in dynamic few-shot text classification, where labeled data is scarce, and new target labels frequently appear based on application needs. Recently, large language models (LLMs) have shown promise due to their extensive pretraining and contextual understanding ability. Current approaches provide LLMs with text inputs, candidate labels, and additional side information (e.g., descriptions) to classify texts. However, their effectiveness is hindered by the increased input size and the noise introduced through side information processing. To address these limitations, we propose a graph-based online retrieval-augmented generation framework, namely GORAG, for dynamic few-shot text classification. Rather than treating each input independently, GORAG constructs and maintains a weighted graph by extracting side information across all target texts. In this graph, text keywords and labels are represented as nodes, with edges indicating the correlations between them. To model these correlations, GORAG employs an edge weighting mechanism to prioritize the importance and reliability of extracted information and dynamically retrieves relevant context using a minimum-cost spanning tree tailored for each text input. Empirical evaluations demonstrate that GORAG outperforms existing approaches by providing more comprehensive and precise contextual information.

Figures (6)

• Figure 1: An example of the DFSTC task with two rounds. In round 1, an example text is classified as Machine Learning. In round 2, an example text is classified as Computer Vision.
• Figure 2: An overview of GraphRAG edge2024local. It first extracts and indexes entities and relations as graph nodes and edges. Then it performs Top-K similarity-based retrieval for query-related subgraphs. Finally, it classifies texts based on the subgraph summary and the query text.
• Figure 3: An example of the constructed weighted graph, where red nodes denote the label nodes and black nodes denote the keyword nodes. The lower the edge cost is, the more related the keyword is to the respective label.
• Figure 4: An overview of GORAG in each round. In Part 1, GORAG constructs a weighted graph based on keywords extracted from the few-shot training data. In Part 2, GORAG performs adaptive graph retrieval and outputs the candidate labels, which is a subset of the original target label set. In Part 3, GORAG first classifies texts into one of the candidate labels, and then applies online indexing to update the graph with newly extracted nodes from query text.
• Figure 5: Experiment result of WOS dataset under 1-shot and 5-shot settings for at most 8 rounds, and model Training/Indexing time cost on two datasets under 1-shot setting.

Theorems & Definitions (3)

• definition 1: Adaptive Candidate label Generation Problem
• theorem 1
• proof