Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction

Alexander Brinkmann, Roee Shraga, Christian Bizer

TL;DR

This work investigates how large language models can extract attribute-value pairs from unstructured product descriptions, addressing the data inefficiency and poor generalization of traditional PLMs. It systematically evaluates zero-shot and few-shot prompting across multiple target-schema representations, including textual and JSON formats, and demonstrates that GPT-4 achieves the highest average F1 around 85 with schema-informed prompts and demonstrations, while open-source Llama-3-70B remains competitive. The study shows LLMs are more training-data-efficient than PLMs, with GPT-4 outperforming baselines by notable margins, though fine-tuned GPT-3.5 can match GPT-4 but may lose generalization to unseen values. The results highlight ExtractGPT as a practical, data-efficient approach for evolving AVE tasks and provide guidance on when to prefer prompt-based LLM methods versus fine-tuning in production settings.

Abstract

E-commerce platforms require structured product data in the form of attribute-value pairs to offer features such as faceted product search or attribute-based product comparison. However, vendors often provide unstructured product descriptions, necessitating the extraction of attribute-value pairs from these texts. BERT-based extraction methods require large amounts of task-specific training data and struggle with unseen attribute values. This paper explores using large language models (LLMs) as a more training-data efficient and robust alternative. We propose prompt templates for zero-shot and few-shot scenarios, comparing textual and JSON-based target schema representations. Our experiments show that GPT-4 achieves the highest average F1-score of 85% using detailed attribute descriptions and demonstrations. Llama-3-70B performs nearly as well, offering a competitive open-source alternative. GPT-4 surpasses the best PLM baseline by 5% in F1-score. Fine-tuning GPT-3.5 increases the performance to the level of GPT-4 but reduces the model's ability to generalize to unseen attribute values.

ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction

TL;DR

This work investigates how large language models can extract attribute-value pairs from unstructured product descriptions, addressing the data inefficiency and poor generalization of traditional PLMs. It systematically evaluates zero-shot and few-shot prompting across multiple target-schema representations, including textual and JSON formats, and demonstrates that GPT-4 achieves the highest average F1 around 85 with schema-informed prompts and demonstrations, while open-source Llama-3-70B remains competitive. The study shows LLMs are more training-data-efficient than PLMs, with GPT-4 outperforming baselines by notable margins, though fine-tuned GPT-3.5 can match GPT-4 but may lose generalization to unseen values. The results highlight ExtractGPT as a practical, data-efficient approach for evolving AVE tasks and provide guidance on when to prefer prompt-based LLM methods versus fine-tuning in production settings.

Abstract

E-commerce platforms require structured product data in the form of attribute-value pairs to offer features such as faceted product search or attribute-based product comparison. However, vendors often provide unstructured product descriptions, necessitating the extraction of attribute-value pairs from these texts. BERT-based extraction methods require large amounts of task-specific training data and struggle with unseen attribute values. This paper explores using large language models (LLMs) as a more training-data efficient and robust alternative. We propose prompt templates for zero-shot and few-shot scenarios, comparing textual and JSON-based target schema representations. Our experiments show that GPT-4 achieves the highest average F1-score of 85% using detailed attribute descriptions and demonstrations. Llama-3-70B performs nearly as well, offering a competitive open-source alternative. GPT-4 surpasses the best PLM baseline by 5% in F1-score. Fine-tuning GPT-3.5 increases the performance to the level of GPT-4 but reduces the model's ability to generalize to unseen attribute values.
Paper Structure (15 sections, 4 figures, 11 tables)

This paper contains 15 sections, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Experimental Setup
  4. Datasets
  5. Large Language Models
  6. Evaluation Metrics
  7. Zero-Shot Prompt Engineering
  8. Prompt Templates
  9. Representation of the Target Schema
  10. Using Training Data
  11. Example Values
  12. In-Context Demonstrations
  13. Fine-Tuning
  14. Comparison to PLM-based Baselines
  15. Conclusions

Figures (4)

  • Figure 1: An example product title with tagged attribute-value pairs. Vendors include product attribute values in the title to enhance visibility.
  • Figure 2: Zero-shot prompt templates list and schema.
  • Figure 3: Target schema representation formats (a) textual, (b) compact and (c) json.
  • Figure 4: Prompt template for in-context learning.