FLAME: Self-Supervised Low-Resource Taxonomy Expansion using Large Language Models

TL;DR

FLAME introduces a self-supervised taxonomy expansion framework that leverages few-shot prompting and reinforcement learning to inject new concepts into an expert-curated seed taxonomy. By constructing taxonomy-aware prompts from diverse global and local samples and fine-tuning a low-rank LLM with PPO, FLAME achieves strong hypernym prediction while maintaining taxonomy coherence. Across three SemEval-2016 benchmarks, FLAME outperforms eight baselines in both accuracy and Wu & Palmer similarity, with ablations confirming the importance of global/local context and an optimal number of prompt instances. The approach offers a scalable, resource-efficient pathway to keep taxonomies up-to-date in real-world domains, and suggests future work on refining lexical rewards and extending relation augmentation.

Abstract

Taxonomies represent an arborescence hierarchical structure that establishes relationships among entities to convey knowledge within a specific domain. Each edge in the taxonomy signifies a hypernym-hyponym relationship. Taxonomies find utility in various real-world applications, such as e-commerce search engines and recommendation systems. Consequently, there arises a necessity to enhance these taxonomies over time. However, manually curating taxonomies with neoteric data presents challenges due to limitations in available human resources and the exponential growth of data. Therefore, it becomes imperative to develop automatic taxonomy expansion methods. Traditional supervised taxonomy expansion approaches encounter difficulties stemming from limited resources, primarily due to the small size of existing taxonomies. This scarcity of training data often leads to overfitting. In this paper, we propose FLAME, a novel approach for taxonomy expansion in low-resource environments by harnessing the capabilities of large language models that are trained on extensive real-world knowledge. LLMs help compensate for the scarcity of domain-specific knowledge. Specifically, FLAME leverages prompting in few-shot settings to extract the inherent knowledge within the LLMs, ascertaining the hypernym entities within the taxonomy. Furthermore, it employs reinforcement learning to fine-tune the large language models, resulting in more accurate predictions. Experiments on three real-world benchmark datasets demonstrate the effectiveness of FLAME in real-world scenarios, achieving a remarkable improvement of 18.5% in accuracy and 12.3% in Wu & Palmer metric over eight baselines. Furthermore, we elucidate the strengths and weaknesses of FLAME through an extensive case study, error analysis and ablation studies on the benchmarks.

Figures (4)

• Figure 1: A panoramic view of FLAME for the taxonomy expansion task to include new concepts such as "zinc". A taxonomy-centric prompt is used to fine-tune the LLM (LLaMA-2 7B) using the text-alignment reward function.
• Figure 2: An illustration of FLAME framework. Given a seed taxonomy, a $k$-shot prompt is generated (Section \ref{['sec:taxo_prompt']}) through the taxonomy prompt template (Section \ref{['subsec:prompt']}) by aggregating nodes into clusters through hierarchical clustering and generating global sample pool $S_g$ (Section \ref{['subsec:global']}) and local sample pool $S_l$ (Section \ref{['subsec:local']}).
• Figure 3: Performance of FLAME for the different number of examples ($k$) included in the prompt.
• Figure 4: Performance of FLAME without local and global samples included in the prompt. The horizontal lines represent the performance of FLAME when both global and local samples are included.