Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

More Discriminative Sentence Embeddings via Semantic Graph Smoothing

Chakib Fettal, Lazhar Labiod, Mohamed Nadif

TL;DR

The paper introduces semantic graph smoothing to obtain more discriminative sentence embeddings in an unsupervised setting by smoothing pretrained embeddings over a cosine similarity based $k$-NN graph using polynomial graph filters. By applying four propagation schemes (SGC, S²GC, APPNP, DGC) and tuning a small set of hyperparameters, the approach improves clustering and classification performance across eight benchmark datasets. Empirical results show systematic gains and statistical significance over filterless baselines, highlighting the utility of graph signal processing for enhancing text representations without labels. The method offers a practical and scalable way to boost both unsupervised and supervised document categorization tasks, with clear tradeoffs in computation and parameter tuning.

Abstract

This paper explores an empirical approach to learn more discriminantive sentence representations in an unsupervised fashion. Leveraging semantic graph smoothing, we enhance sentence embeddings obtained from pretrained models to improve results for the text clustering and classification tasks. Our method, validated on eight benchmarks, demonstrates consistent improvements, showcasing the potential of semantic graph smoothing in improving sentence embeddings for the supervised and unsupervised document categorization tasks.

More Discriminative Sentence Embeddings via Semantic Graph Smoothing

TL;DR

The paper introduces semantic graph smoothing to obtain more discriminative sentence embeddings in an unsupervised setting by smoothing pretrained embeddings over a cosine similarity based -NN graph using polynomial graph filters. By applying four propagation schemes (SGC, S²GC, APPNP, DGC) and tuning a small set of hyperparameters, the approach improves clustering and classification performance across eight benchmark datasets. Empirical results show systematic gains and statistical significance over filterless baselines, highlighting the utility of graph signal processing for enhancing text representations without labels. The method offers a practical and scalable way to boost both unsupervised and supervised document categorization tasks, with clear tradeoffs in computation and parameter tuning.

Abstract

This paper explores an empirical approach to learn more discriminantive sentence representations in an unsupervised fashion. Leveraging semantic graph smoothing, we enhance sentence embeddings obtained from pretrained models to improve results for the text clustering and classification tasks. Our method, validated on eight benchmarks, demonstrates consistent improvements, showcasing the potential of semantic graph smoothing in improving sentence embeddings for the supervised and unsupervised document categorization tasks.
Paper Structure (14 sections, 6 equations, 1 figure, 4 tables)

This paper contains 14 sections, 6 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Background: Graph Filtering and Smoothing
  3. Graph Signals
  4. Graph Filters
  5. Proposed Methodology: Smoothing Sentence Embeddings
  6. Experiments
  7. Datasets and Metrics
  8. Experimental Settings
  9. Experimental Results
  10. Clustering Results
  11. Classification Results
  12. Statistical Significance Testing
  13. Conclusion
  14. Limitations

Figures (1)

  • Figure 1: Bonferroni-Dunn average rank test at a confidence level of 95%.