Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-tuning Large Language Models for Entity Matching

Aaron Steiner, Ralph Peeters, Christian Bizer

TL;DR

This work demonstrates that fine-tuning small LLMs for entity matching can close the gap with larger models while offering substantial cost savings, especially when guided by structured explanations. It introduces two core dimensions—training-example representation (including explanations) and example selection/generation—and systematically analyzes their effects on in-domain and cross-domain generalization. Structured explanations consistently boost in-domain generalization for several models, while the benefits of example generation and filtration are model-dependent, sometimes yielding large gains for Llama-8B but not for GPT-4o. Overall, the study highlights a practical path to scalable, cost-efficient entity matching with LLMs, while pointing to remaining challenges in cross-domain transfer and arguing for refined data-augmentation strategies in future work.

Abstract

Generative large language models (LLMs) are a promising alternative to pre-trained language models for entity matching due to their high zero-shot performance and ability to generalize to unseen entities. Existing research on using LLMs for entity matching has focused on prompt engineering and in-context learning. This paper explores the potential of fine-tuning LLMs for entity matching. We analyze fine-tuning along two dimensions: 1) the representation of training examples, where we experiment with adding different types of LLM-generated explanations to the training set, and 2) the selection and generation of training examples using LLMs. In addition to the matching performance on the source dataset, we investigate how fine-tuning affects the models ability to generalize to other in-domain datasets as well as across topical domains. Our experiments show that fine-tuning significantly improves the performance of the smaller models while the results for the larger models are mixed. Fine-tuning also improves the generalization to in-domain datasets while hurting cross-domain transfer. We show that adding structured explanations to the training set has a positive impact on the performance of three out of four LLMs, while the proposed example selection and generation methods, only improve the performance of Llama 3.1 8B while decreasing the performance of GPT-4o-mini.

Fine-tuning Large Language Models for Entity Matching

TL;DR

This work demonstrates that fine-tuning small LLMs for entity matching can close the gap with larger models while offering substantial cost savings, especially when guided by structured explanations. It introduces two core dimensions—training-example representation (including explanations) and example selection/generation—and systematically analyzes their effects on in-domain and cross-domain generalization. Structured explanations consistently boost in-domain generalization for several models, while the benefits of example generation and filtration are model-dependent, sometimes yielding large gains for Llama-8B but not for GPT-4o. Overall, the study highlights a practical path to scalable, cost-efficient entity matching with LLMs, while pointing to remaining challenges in cross-domain transfer and arguing for refined data-augmentation strategies in future work.

Abstract

Generative large language models (LLMs) are a promising alternative to pre-trained language models for entity matching due to their high zero-shot performance and ability to generalize to unseen entities. Existing research on using LLMs for entity matching has focused on prompt engineering and in-context learning. This paper explores the potential of fine-tuning LLMs for entity matching. We analyze fine-tuning along two dimensions: 1) the representation of training examples, where we experiment with adding different types of LLM-generated explanations to the training set, and 2) the selection and generation of training examples using LLMs. In addition to the matching performance on the source dataset, we investigate how fine-tuning affects the models ability to generalize to other in-domain datasets as well as across topical domains. Our experiments show that fine-tuning significantly improves the performance of the smaller models while the results for the larger models are mixed. Fine-tuning also improves the generalization to in-domain datasets while hurting cross-domain transfer. We show that adding structured explanations to the training set has a positive impact on the performance of three out of four LLMs, while the proposed example selection and generation methods, only improve the performance of Llama 3.1 8B while decreasing the performance of GPT-4o-mini.
Paper Structure (15 sections, 1 equation, 4 figures, 6 tables)

This paper contains 15 sections, 1 equation, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Experimental Setup
  3. Standard Fine-Tuning
  4. Effectiveness
  5. Generalization
  6. Dimension 1: Example Representation
  7. Effectiveness
  8. Generalization
  9. Dimension 2: Example Selection and Generation
  10. Training Set Filtration
  11. Example Generation
  12. Error-based Example Selection
  13. Cost Analysis
  14. Related work
  15. Conclusion

Figures (4)

  • Figure 1: Fine-tuning and inference setup including explanation generation, example generation, and filtration.
  • Figure 2: Representation of training examples for standard fine-tuning.
  • Figure 3: Training example containing a textual explanation.
  • Figure 4: Training example containing a Structured explanation.