MoEMeta: Mixture-of-Experts Meta Learning for Few-Shot Relational Learning

TL;DR

MoEMeta tackles few-shot relational learning in knowledge graphs by separating globally shared meta-knowledge from task-specific contexts. It introduces a mixture-of-experts based relational prototype learner to generalize across tasks and a task-tailored local adaptation mechanism to fine-tune embeddings for each task. Across Nell-One, Wiki-One, and FB15K-One benchmarks, MoEMeta achieves state-of-the-art results, with ablations confirming the value of both global prototypes and local adaptation. The approach offers a scalable and efficient solution for rapid reasoning over unseen relations under limited supervision, evidenced by improved top-k accuracy and mean reciprocal rank.

Abstract

Few-shot knowledge graph relational learning seeks to perform reasoning over relations given only a limited number of training examples. While existing approaches largely adopt a meta-learning framework for enabling fast adaptation to new relations, they suffer from two key pitfalls. First, they learn relation meta-knowledge in isolation, failing to capture common relational patterns shared across tasks. Second, they struggle to effectively incorporate local, task-specific contexts crucial for rapid adaptation. To address these limitations, we propose MoEMeta, a novel meta-learning framework that disentangles globally shared knowledge from task-specific contexts to enable both effective generalization and rapid adaptation. MoEMeta introduces two key innovations: (i) a mixture-of-experts (MoE) model that learns globally shared relational prototypes to enhance generalization, and (ii) a task-tailored adaptation mechanism that captures local contexts for fast task-specific adaptation. By balancing global generalization with local adaptability, MoEMeta significantly advances few-shot relational learning. Extensive experiments and analyses on three KG benchmarks demonstrate that MoEMeta consistently outperforms existing baselines, achieving state-of-the-art performance.

Figures (4)

• Figure 1: The computation of the support loss. The support loss $\mathcal{L}(\mathcal{S}_r)$ is used to update relation-meta $\mathbf{R}_{\mathcal{T}_r}$, which is applied to the query set for model optimization.
• Figure 2: Two gating value distributions over 32 experts generated by our MoE-based meta-knowledge learner. The upper figure shows that three similar relations exhibit similar gating value distributions, suggesting that a similar set of experts are activated. In contrast, the lower figure demonstrates that for relations with distinct relational patterns, the activated experts differ significantly. This highlights the presence of shared relational patterns, underscoring the necessity of learning relation prototypes.
• Figure 3: Performance analysis under 5-shot setting on Nell-One for1-N, N-1 and N-N relation types.
• Figure 4: Hyperparameter sensitivity study of MoE under 5-shot setting on Nell-One.