Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyperBox: A Supervised Approach for Hypernym Discovery using Box Embeddings

Maulik Parmar, Apurva Narayan

TL;DR

HyperBox introduces a box-embedding-based approach for hypernym discovery, encoding hypernym relations with two axis-aligned boxes $h^{(1)},h^{(2)} \in \mathbb{R}^d$ and representing each word by a base embedding $\mathbf{w_i}$ plus a translational bump $\mathbf{b_i}$. For a given hyponym-hypernym pair, final embeddings are computed as $\mathbf{w_i^{h}}=(\mathbf{w_i}+\mathbf{b_j})$ and $\mathbf{w_j^{h}}=(\mathbf{w_j}+\mathbf{b_i})$, and the pair is valid if each final embedding lies in its corresponding box; the pair score uses a BoxE-inspired distance dist, with the overall score $score(h(w_1,w_2)) = \sum_{i=1}^{2} ||\mathrm{dist}(\mathbf{w_i^{h(w_1,w_2)}}, \mathbf{h}^{(i)})||_2$. Training optimizes the embeddings and boxes with a negative-sampling objective $L = -\log\sigma(\gamma - dist(v;q)) - \sum_{i=1}^{k} (1/k)\log\sigma(dist(\acute{v}_i;q) - \gamma)$ using Adam. Evaluations on SemEval-2018 Task9 in medical and music domains show HyperBox outperforms most baselines on metrics like MRR, MAP, and P@k, while demonstrating strong generalization to unseen hypernym pairs with limited training data. The results underscore the effectiveness of box embeddings in capturing antisymmetry and hierarchy, with potential for hybridization with Hearst-pattern cues for broader hypernym discovery applications.

Abstract

Hypernymy plays a fundamental role in many AI tasks like taxonomy learning, ontology learning, etc. This has motivated the development of many automatic identification methods for extracting this relation, most of which rely on word distribution. We present a novel model HyperBox to learn box embeddings for hypernym discovery. Given an input term, HyperBox retrieves its suitable hypernym from a target corpus. For this task, we use the dataset published for SemEval 2018 Shared Task on Hypernym Discovery. We compare the performance of our model on two specific domains of knowledge: medical and music. Experimentally, we show that our model outperforms existing methods on the majority of the evaluation metrics. Moreover, our model generalize well over unseen hypernymy pairs using only a small set of training data.

HyperBox: A Supervised Approach for Hypernym Discovery using Box Embeddings

TL;DR

HyperBox introduces a box-embedding-based approach for hypernym discovery, encoding hypernym relations with two axis-aligned boxes and representing each word by a base embedding plus a translational bump . For a given hyponym-hypernym pair, final embeddings are computed as and , and the pair is valid if each final embedding lies in its corresponding box; the pair score uses a BoxE-inspired distance dist, with the overall score . Training optimizes the embeddings and boxes with a negative-sampling objective using Adam. Evaluations on SemEval-2018 Task9 in medical and music domains show HyperBox outperforms most baselines on metrics like MRR, MAP, and P@k, while demonstrating strong generalization to unseen hypernym pairs with limited training data. The results underscore the effectiveness of box embeddings in capturing antisymmetry and hierarchy, with potential for hybridization with Hearst-pattern cues for broader hypernym discovery applications.

Abstract

Hypernymy plays a fundamental role in many AI tasks like taxonomy learning, ontology learning, etc. This has motivated the development of many automatic identification methods for extracting this relation, most of which rely on word distribution. We present a novel model HyperBox to learn box embeddings for hypernym discovery. Given an input term, HyperBox retrieves its suitable hypernym from a target corpus. For this task, we use the dataset published for SemEval 2018 Shared Task on Hypernym Discovery. We compare the performance of our model on two specific domains of knowledge: medical and music. Experimentally, we show that our model outperforms existing methods on the majority of the evaluation metrics. Moreover, our model generalize well over unseen hypernymy pairs using only a small set of training data.
Paper Structure (14 sections, 7 equations, 2 figures, 4 tables)

This paper contains 14 sections, 7 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. HyperBox: Proposed Method
  4. Hypernym Discovery using HyperBox
  5. Scoring Function
  6. Training Objective
  7. Experiments
  8. Datasets
  9. Evaluation Metrics
  10. Experimental Setup
  11. Results
  12. Conclusion
  13. Bibliographical References
  14. Language Resource References

Figures (2)

  • Figure 1: Examples of Hypernym-Hyponym pairs
  • Figure 2: An example HyperBox model for three words $w_1$, $w_2$, $w_3$ in $R^2$ for hypernymy pairs $h(w_2,w_3)$, $h(w_2,w_1)$ and $h(w_1,w_3)$. The hypernymy relation is encoded by box embeddings $\mathbf{h^{(1)}}$ and $\mathbf{h^{(2)}}$. Every word $w_i$ has an embedding $\mathbf{w_i}$, and $\mathbf{b_i}$ which defines a bump on other words, as shown with distinct colors.