Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-supervised Topic Taxonomy Discovery in the Box Embedding Space

Yuyin Lu, Hegang Chen, Pengbo Mao, Yanghui Rao, Haoran Xie, Fu Lee Wang, Qing Li

TL;DR

This work introduces BoxTM, a self-supervised topic taxonomy discovery method that embeds words and topics as axis-aligned boxes, enabling explicit modeling of semantic scopes and asymmetric hierarchical relations. It combines a VAE-based document generative process with recursive clustering of topic boxes to mine upper-level topics, supported by word- and topic-level self-supervised constraints that tie box volumes to co-occurrence statistics and hierarchical containment. The approach delivers state-of-the-art performance across intrinsic quality (coherence, diversity, hierarchical coherence), extrinsic hierarchical clustering, and human evaluation, while providing interpretable, adaptive taxonomy depths. The results demonstrate that box embeddings, coupled with recursive clustering and scoped constraints, yield higher-quality, scalable topic taxonomies suitable for navigation, search, and downstream textual analytics.

Abstract

Topic taxonomy discovery aims at uncovering topics of different abstraction levels and constructing hierarchical relations between them. Unfortunately, most of prior work can hardly model semantic scopes of words and topics by holding the Euclidean embedding space assumption. What's worse, they infer asymmetric hierarchical relations by symmetric distances between topic embeddings. As a result, existing methods suffer from problems of low-quality topics at high abstraction levels and inaccurate hierarchical relations. To alleviate these problems, this paper develops a Box embedding-based Topic Model (BoxTM) that maps words and topics into the box embedding space, where the asymmetric metric is defined to properly infer hierarchical relations among topics. Additionally, our BoxTM explicitly infers upper-level topics based on correlation between specific topics through recursive clustering on topic boxes. Finally, extensive experiments validate high-quality of the topic taxonomy learned by BoxTM.

Self-supervised Topic Taxonomy Discovery in the Box Embedding Space

TL;DR

This work introduces BoxTM, a self-supervised topic taxonomy discovery method that embeds words and topics as axis-aligned boxes, enabling explicit modeling of semantic scopes and asymmetric hierarchical relations. It combines a VAE-based document generative process with recursive clustering of topic boxes to mine upper-level topics, supported by word- and topic-level self-supervised constraints that tie box volumes to co-occurrence statistics and hierarchical containment. The approach delivers state-of-the-art performance across intrinsic quality (coherence, diversity, hierarchical coherence), extrinsic hierarchical clustering, and human evaluation, while providing interpretable, adaptive taxonomy depths. The results demonstrate that box embeddings, coupled with recursive clustering and scoped constraints, yield higher-quality, scalable topic taxonomies suitable for navigation, search, and downstream textual analytics.

Abstract

Topic taxonomy discovery aims at uncovering topics of different abstraction levels and constructing hierarchical relations between them. Unfortunately, most of prior work can hardly model semantic scopes of words and topics by holding the Euclidean embedding space assumption. What's worse, they infer asymmetric hierarchical relations by symmetric distances between topic embeddings. As a result, existing methods suffer from problems of low-quality topics at high abstraction levels and inaccurate hierarchical relations. To alleviate these problems, this paper develops a Box embedding-based Topic Model (BoxTM) that maps words and topics into the box embedding space, where the asymmetric metric is defined to properly infer hierarchical relations among topics. Additionally, our BoxTM explicitly infers upper-level topics based on correlation between specific topics through recursive clustering on topic boxes. Finally, extensive experiments validate high-quality of the topic taxonomy learned by BoxTM.
Paper Structure (27 sections, 16 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 27 sections, 16 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Document Generation-based Methods
  4. Clustering-based Methods
  5. Supervised Methods
  6. Background Knowledge
  7. Proposed Method
  8. Document Generative Process
  9. Recursive Topic Clustering
  10. Semantic Scope Modeling
  11. Word-level Constraint
  12. Topic-level Constraint
  13. Learning Strategy
  14. Experiments
  15. Experimental Settings
  16. ...and 12 more sections

Figures (8)

  • Figure 1: The topic taxonomy discovery processes in the point embedding space (a-c) and the box embedding space (d-f) of most existing HNTMs and the proposed BoxTM, respectively.
  • Figure 2: The main framework of BoxTM.
  • Figure 3: The C*D scores at each level of BoxTM and baselines on NYT.
  • Figure 4: Illustration of the human evaluation on the NYT dataset: an example of the topic intrusion task (left) and the average precision (%) of our BoxTM and strong baselines (right).
  • Figure 5: Illustration of the partial topic taxonomy learned by BoxTM on arXiv (a) and NYT (b).
  • ...and 3 more figures

Theorems & Definitions (2)

  • Definition 1: box embedding
  • Definition 2: box operations