GHTM: A Graph-based Hybrid Topic Modeling Approach with a Benchmark Dataset for the Low-Resource Bengali Language

Abstract

Topic modeling is a Natural Language Processing (NLP) technique used to discover latent themes and abstract topics from text corpora by grouping co-occurring keywords. Although widely researched in English, topic modeling remains understudied in Bengali due to a lack of adequate resources and initiatives. Existing Bengali topic modeling research lacks standardized evaluation frameworks with comprehensive baselines and diverse datasets, exploration of modern methodological approaches, and reproducible implementations, with only three Bengali-specific architectures proposed to date. To address these gaps, this study presents a comprehensive evaluation of traditional and contemporary topic modeling approaches across three Bengali datasets and introduces GHTM (Graph-based Hybrid Topic Model), a novel architecture that strategically integrates TF-IDF-weighted GloVe embeddings, Graph Convolutional Networks (GCN), and Non-negative Matrix Factorization (NMF). GHTM represents text documents using hybrid TF-IDF-weighted GloVe embeddings. It builds a document-similarity graph and leverages GCN to refine the representations through neighborhood aggregation. Then, it finally decomposes the refined representations using NMF to extract interpretable topics. Experimental results demonstrate that GHTM achieves superior topic coherence (NPMI: 0.27-0.28) and diversity compared to existing methods while maintaining computational efficiency across datasets of varying scales. The model also demonstrates strong cross-lingual generalization, outperforming established graph-based models on the English 20Newsgroups benchmark. Additionally, we introduce NCTBText, a diverse Bengali textbook-based dataset comprising 8,650 text documents, curated from eight subject areas, providing much-needed topical diversity beyond newspaper-centric Bengali corpora and serving as a benchmark for future research.

Figures (10)

• Figure 1: Overview Diagram of Graph Hybrid Topic Model (GHTM) architecture. a) Text Vectorization: This stage converts the text documents into a sum of TF-IDF weighted GloVe embeddings. b) Graph-based Representation Learning: This stage generates a KNN graph, and separates it into sub-graphs. Document vectors, represented as nodes in the graph, are enriched by the GCN layers, resulting in the output matrix $\mathbf{Z}$. The output graph, as shown in the diagram, densely clusters related nodes, which are now communally informed. c) Non-negative Matrix Factorization: This stage accomplishes topic modeling by first performing an absolute value transformation on $\mathbf{Z}$ and then factorizing it into $\mathbf{W}$ and $\mathbf{H}$. d) Topic Representation: Finally, this stage identifies the top $R$ representative documents from $\mathbf{W}$ and chooses a collection of keywords to represent the topics.
• Figure 2: Class Distribution of NCTBText. The 8,650 texts in the dataset are distributed across eight subject categories as follows, in descending order: Religion (1,999), Science (1,712), Bengali (1,399), Bangladesh and Global Studies - BGS (1,090), Business (656), Home Science (607), Agriculture (600), and Information and Communication Technology - ICT (587).
• Figure 3: Two-dimensional t-SNE projection of NCTBText. The document embeddings of the NCTBText dataset are represented in two dimensions using t-SNE. Each point depicts a document, colored in accordance with its true class label. Distinct clusters show that the categories are well-separable in the embedding space.
• Figure 4: Common and Rare Words in NCTBText. Frequency distribution of the common and rare words in the NCTBText dataset. Common words that are frequently found on the dataset are represented in purple on the right, while the words with low frequencies are represented in pink on the left.
• Figure 5: Zipf’s Law Validation of NCTBText. Log-log graph of word frequency against rank. In accordance with Zipf’s law, this distribution indicates that in the NCTBText dataset, a small number of words occur very frequently while the majority of words appear rarely.