Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised MMEA

Yunpeng Hong, Chenyang Bu, Jie Zhang, Yi He, Di Wu, Xindong Wu

TL;DR

This work proposes PSQE (Pseudo-Seed Quality Enhancement) to improve the precision and graph coverage balance of pseudo seeds via multimodal information and clustering-resampling and shows that PSQE as a plug-and-play module can improve the performance of baselines by considerable margins.

Abstract

Multimodal Entity Alignment (MMEA) aims to identify equivalent entities across different data modalities, enabling structural data integration that in turn improves the performance of various large language model applications. To lift the requirement of labeled seed pairs that are difficult to obtain, recent methods shifted to an unsupervised paradigm using pseudo-alignment seeds. However, unsupervised entity alignment in multimodal settings remains underexplored, mainly because the incorporation of multimodal information often results in imbalanced coverage of pseudo-seeds within the knowledge graph. To overcome this, we propose PSQE (Pseudo-Seed Quality Enhancement) to improve the precision and graph coverage balance of pseudo seeds via multimodal information and clustering-resampling. Theoretical analysis reveals the impact of pseudo seeds on existing contrastive learning-based MMEA models. In particular, pseudo seeds can influence the attraction and the repulsion terms in contrastive learning at once, whereas imbalanced graph coverage causes models to prioritize high-density regions, thereby weakening their learning capability for entities in sparse regions. Experimental results validate our theoretical findings and show that PSQE as a plug-and-play module can improve the performance of baselines by considerable margins.

PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised MMEA

TL;DR

This work proposes PSQE (Pseudo-Seed Quality Enhancement) to improve the precision and graph coverage balance of pseudo seeds via multimodal information and clustering-resampling and shows that PSQE as a plug-and-play module can improve the performance of baselines by considerable margins.

Abstract

Multimodal Entity Alignment (MMEA) aims to identify equivalent entities across different data modalities, enabling structural data integration that in turn improves the performance of various large language model applications. To lift the requirement of labeled seed pairs that are difficult to obtain, recent methods shifted to an unsupervised paradigm using pseudo-alignment seeds. However, unsupervised entity alignment in multimodal settings remains underexplored, mainly because the incorporation of multimodal information often results in imbalanced coverage of pseudo-seeds within the knowledge graph. To overcome this, we propose PSQE (Pseudo-Seed Quality Enhancement) to improve the precision and graph coverage balance of pseudo seeds via multimodal information and clustering-resampling. Theoretical analysis reveals the impact of pseudo seeds on existing contrastive learning-based MMEA models. In particular, pseudo seeds can influence the attraction and the repulsion terms in contrastive learning at once, whereas imbalanced graph coverage causes models to prioritize high-density regions, thereby weakening their learning capability for entities in sparse regions. Experimental results validate our theoretical findings and show that PSQE as a plug-and-play module can improve the performance of baselines by considerable margins.
Paper Structure (42 sections, 1 theorem, 22 equations, 7 figures, 6 tables)

This paper contains 42 sections, 1 theorem, 22 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Entity Alignment
  4. Balanced Data Distribution for Contrastive Learning
  5. Preliminaries
  6. Multimodal Entity Alignment
  7. Contrastive Learning Loss
  8. Method
  9. Stage I: Multimodal Fusion & Cluster Sampling
  10. Multimodal Fusion
  11. Cluster Sampling
  12. Stage II: Global Sampling & Error Correction
  13. Global Sampling
  14. Multimodal Information Correction
  15. Stage III: Neighborhood Expansion & Rechecking
  16. ...and 27 more sections

Key Result

Theorem 1

Assuming that a regularization function is applied to the embedding representation of an entity, let $S = \{ (h_1^1,h_2^1), \ldots, (h_1^n,h_2^n)\}$ be the set of pseudo seeds where $h_1^n$ denotes the vector representation of entity $n$ in knowledge graph $1$ and $||h|| = 1$. Then, the lower bound where ${\mathcal{N}}_i^{ng}$ denotes the set of negative samples for entity i, and $D=|B|-1$ corres

Figures (7)

  • Figure 1: Comparative analysis of three types of pseudo seed generation on the FR-EN dataset for Multimodal Entity Alignment (MMEA): (1) Type I: Unimodal, (2) Type II: Multimodal, and (3) Type III: Distribution-Aware Multimodal. An interesting phenomenon is observed: Type II achieves higher precision than Type I but performs worse in the downstream MMEA task. In contrast, Type III, due to its better distribution balance, yields the best performance.
  • Figure 2: Overall framework of PSQE. PSQE optimizes the precision and the graph coverage balance (distribution) of pseudo-alignment seeds in three stages to enhance their quality.
  • Figure 3: Example of an attraction term for Remark 1: $x$ and $y$ are correct entity pairs. Wrongly aligned seeds can push away vector representations from the correct entity, resulting in variability from the vector of the correct entity.
  • Figure 4: Examples of a repulsion term of Remark 2: $x^-$ is a neighboring entity, and red arrows indicate new negative samples. Negative samples were not added to $x_7^-$, $x_9^-$ in the left figure. Unbalanced graph coverage leads to focusing on $x_1^-$, $x_2^-$, and $x_3^-$ during optimization and under-optimization of entities in the region around $x_8^-$, which are difficult to distinguish, e.g., the bottom right corner of Fig. (a).
  • Figure 5: Case study of the results of MEAformer's comparison on the unsupervised setting of PSQE vs. UVP on the JA-EN dataset, where correct entity pairs are marked with '✓' and incorrect ones with '×'.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Remark 1
  • Remark 2