How Does A Text Preprocessing Pipeline Affect Ontology Matching?

TL;DR

The paper investigates how a classical text preprocessing pipeline affects ontology matching, revealing that Tokenisation and Normalisation enhance precision and recall while Stop Words Removal and Stemming/Lemmatisation often harm performance. It introduces two repair strategies—an ad hoc logic-based preprocessor and a post hoc LLM-based validator—to mitigate Phase 2 weaknesses and demonstrates substantial gains in accuracy across 49 alignments from 8 OAEI tracks. The work also analyzes prompting strategies for LLMs and demonstrates that combining traditional preprocessing with LLM-based validation yields more robust seed mappings than either approach alone. Limitations include imperfect gold standards and focus on equivalence mappings, with artifacts and data made available for reproducibility. Overall, the study provides objective guidance on when and how to apply text preprocessing in OM and showcases practical pathways to integrate LLMs without discarding established preprocessing pipelines.

Abstract

The classical text preprocessing pipeline, comprising Tokenisation, Normalisation, Stop Words Removal, and Stemming/Lemmatisation, has been implemented in many systems for ontology matching (OM). However, the lack of standardisation in text preprocessing creates diversity in the mapping results. In this paper, we investigate the effect of the text preprocessing pipeline on 8 Ontology Alignment Evaluation Initiative (OAEI) tracks with 49 distinct alignments. We find that Tokenisation and Normalisation (categorised as Phase 1 text preprocessing) are more effective than Stop Words Removal and Stemming/Lemmatisation (categorised as Phase 2 text preprocessing). We propose two novel approaches to repair unwanted false mappings that occur in Phase 2 text preprocessing. One is an ad hoc logic-based repair approach used before text preprocessing, employing an ontology-specific check to find common words that cause false mappings. The other repair approach is the post hoc large language model (LLM)-based approach, used after text preprocessing, which utilises the strong background knowledge provided by LLMs to repair non-existent and counter-intuitive false mappings. The experimental results indicate that these two approaches can significantly improve the matching correctness and the overall matching performance.

Figures (16)

• Figure 1: An Example of using the text preprocessing pipeline in OM.
• Figure 2: Experiment setup to analyse the effect of text preprocessing pipeline in OM: Tokenisation (T), Normalisation (N), Stop Words Removal (R), and Stemming/Lemmatisation (S/L).
• Figure 3: Number of entities in each alignment across different tracks.
• Figure 4: Frequency distribution of compound words. We exclude entities with more than 15 compound words because their proportion is less than 1%.
• Figure 5: OM evaluation euzenat2007semantic.