GloCTM: Cross-Lingual Topic Modeling via a Global Context Space

TL;DR

Cross-lingual topic modeling often yields misaligned topics due to separate language spaces and limited semantic signals from dictionaries. This work presents GloCTM, a dual-pathway variational autoencoder that learns a Global Context Space via Polyglot Augmentation, uses a unified decoder to enforce cross-lingual topic sharing, and incorporates regularizers KL divergence and Centered Kernel Alignment to ground topics in multilingual embeddings. The approach yields significant gains in cross-lingual topic coherence and alignment across English–Chinese and English–Japanese benchmarks, validated by CNPMI, TU, classification, and LLM-based evaluations. By tightly integrating input enrichment, a shared generative structure, and semantic regularization, GloCTM offers a principled, end-to-end solution for semantically grounded, multilingual topic modeling with practical impact on global text analytics.

Abstract

Cross-lingual topic modeling seeks to uncover coherent and semantically aligned topics across languages - a task central to multilingual understanding. Yet most existing models learn topics in disjoint, language-specific spaces and rely on alignment mechanisms (e.g., bilingual dictionaries) that often fail to capture deep cross-lingual semantics, resulting in loosely connected topic spaces. Moreover, these approaches often overlook the rich semantic signals embedded in multilingual pretrained representations, further limiting their ability to capture fine-grained alignment. We introduce GloCTM (Global Context Space for Cross-Lingual Topic Model), a novel framework that enforces cross-lingual topic alignment through a unified semantic space spanning the entire model pipeline. GloCTM constructs enriched input representations by expanding bag-of-words with cross-lingual lexical neighborhoods, and infers topic proportions using both local and global encoders, with their latent representations aligned through internal regularization. At the output level, the global topic-word distribution, defined over the combined vocabulary, structurally synchronizes topic meanings across languages. To further ground topics in deep semantic space, GloCTM incorporates a Centered Kernel Alignment (CKA) loss that aligns the latent topic space with multilingual contextual embeddings. Experiments across multiple benchmarks demonstrate that GloCTM significantly improves topic coherence and cross-lingual alignment, outperforming strong baselines.

Figures (6)

• Figure 1: Conceptual overview of Cross-Lingual Topic Modeling (CLTM)
• Figure 2: The dual-pathway architecture of GloCTM. The model integrates a Local Pathway for language-specific features with a Global Pathway for unified topic learning. The Global Pathway is uniquely fed by a semantically-enriched input created via Polyglot Augmentation. Alignment is enforced through three core mechanisms: Topic Synchronization ($\beta$), KL Divergence ($\theta_{local} \leftrightarrow \theta_{global}$), and CKA loss with PLM embeddings.
• Figure 3: The Polyglot Augmentation mechanism. An original word ("football") is enriched with its nearest intra-lingual (green) and cross-lingual (red) neighbors. This creates a dense Global BOW vector, preemptively injecting cross-lingual information before the encoding step.
• Figure 4: Formation of a unified 'Food' topic (Topic #2) in GloCTM. The global topic vector is constructed by concatenating the local topic vectors from English ($\beta_2^{(1)}$) and Chinese ($\beta_2^{(2)}$), enforcing semantic alignment by design.
• Figure 5: Classification performance on the three datasets. Except for MCTA and MTAnchor (results reported in infoctm), all scores are averaged over 5 runs.