Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks

Viktor Moskvoretskii, Ekaterina Neminova, Alina Lobanova, Alexander Panchenko, Irina Nikishina

TL;DR

TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA, demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning and explores its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning.

Abstract

In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at https://github.com/VityaVitalich/TaxoLLaMA

TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks

TL;DR

TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA, demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning and explores its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning.

Abstract

In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at https://github.com/VityaVitalich/TaxoLLaMA
Paper Structure (38 sections, 7 figures, 10 tables)

This paper contains 38 sections, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hypernym Discovery
  4. Taxonomy Enrichment
  5. Taxonomy Construction
  6. Lexical Entailment
  7. Methodology
  8. Data Collection
  9. Training Details
  10. Task Adaptation
  11. Experiments
  12. Hypernym Discovery
  13. Taxonomy Enrichment
  14. Taxonomy Construction
  15. Lexical Entailment
  16. ...and 23 more sections

Figures (7)

  • Figure 1: Training procedure of TaxoLLaMA: hypernym relations from the WordNet are linearized and fed into an LLM model. The model aims at generating the correct hypernym(s) as output.
  • Figure 2: Examples with input and output for each task are highlighted by color. Rectangle "hypernym" denotes a word generated by the model; circle means a node from the graph. Confidence score determines the existence of a relationship between the two nodes provided in the input.
  • Figure 3: Experiments for domain and language adaptation on the Hypernym Discovery datasets.
  • Figure 4: Average percentage of error types across Hypernym Discovery and Taxonomy Enrichment datasets.
  • Figure 5: Automatic Evaluation of the MAG datasets with the ChatGPT model. "True" denotes the number of gold answers that ChatGPT preferred over TaxoLLaMA answers; "Predicted" is when ChatGPT preferred TaxoLLaMA output; "Both" and "None" options were also possible answers for ChatGPT.
  • ...and 2 more figures