Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Manifold-Constrained Sentence Embeddings via Triplet Loss: Projecting Semantics onto Spheres, Tori, and Möbius Strips

Vinit K. Chavan

TL;DR

The paper addresses the limitation of Euclidean sentence embeddings by learning representations constrained to non-Euclidean manifolds. It introduces a manifold-constrained framework that projects sentence embeddings onto the unit sphere, torus, or Mobius strip and trains them with a triplet loss L_triplet = sum_i [ ||f(xa_i)-f(xp_i)||^2 - ||f(xa_i)-f(xn_i)||^2 + alpha ]_+. The embedding network combines an encoder, mean pooling, and a projection to the target manifold, with explicit sphere, torus, and Mobius formulations to enforce the geometry. Evaluations on AG News and MBTI show that sphere and Mobius projections improve clustering structure (as measured by Silhouette scores) and classification performance compared to TF-IDF, Word2Vec, and Keras baselines, demonstrating the value of geometry-aware representations. The results motivate geometry-informed NLP models and point to future work on broader manifolds and geodesic-aware learning for multilingual and multimodal tasks.

Abstract

Recent advances in representation learning have emphasized the role of embedding geometry in capturing semantic structure. Traditional sentence embeddings typically reside in unconstrained Euclidean spaces, which may limit their ability to reflect complex relationships in language. In this work, we introduce a novel framework that constrains sentence embeddings to lie on continuous manifolds -- specifically the unit sphere, torus, and Möbius strip -- using triplet loss as the core training objective. By enforcing differential geometric constraints on the output space, our approach encourages the learning of embeddings that are both discriminative and topologically structured. We evaluate our method on benchmark datasets (AG News and MBTI) and compare it to classical baselines including TF-IDF, Word2Vec, and unconstrained Keras-derived embeddings. Our results demonstrate that manifold-constrained embeddings, particularly those projected onto spheres and Möbius strips, significantly outperform traditional approaches in both clustering quality (Silhouette Score) and classification performance (Accuracy). These findings highlight the value of embedding in manifold space -- where topological structure complements semantic separation -- offering a new and mathematically grounded direction for geometric representation learning in NLP.

Manifold-Constrained Sentence Embeddings via Triplet Loss: Projecting Semantics onto Spheres, Tori, and Möbius Strips

TL;DR

The paper addresses the limitation of Euclidean sentence embeddings by learning representations constrained to non-Euclidean manifolds. It introduces a manifold-constrained framework that projects sentence embeddings onto the unit sphere, torus, or Mobius strip and trains them with a triplet loss L_triplet = sum_i [ ||f(xa_i)-f(xp_i)||^2 - ||f(xa_i)-f(xn_i)||^2 + alpha ]_+. The embedding network combines an encoder, mean pooling, and a projection to the target manifold, with explicit sphere, torus, and Mobius formulations to enforce the geometry. Evaluations on AG News and MBTI show that sphere and Mobius projections improve clustering structure (as measured by Silhouette scores) and classification performance compared to TF-IDF, Word2Vec, and Keras baselines, demonstrating the value of geometry-aware representations. The results motivate geometry-informed NLP models and point to future work on broader manifolds and geodesic-aware learning for multilingual and multimodal tasks.

Abstract

Recent advances in representation learning have emphasized the role of embedding geometry in capturing semantic structure. Traditional sentence embeddings typically reside in unconstrained Euclidean spaces, which may limit their ability to reflect complex relationships in language. In this work, we introduce a novel framework that constrains sentence embeddings to lie on continuous manifolds -- specifically the unit sphere, torus, and Möbius strip -- using triplet loss as the core training objective. By enforcing differential geometric constraints on the output space, our approach encourages the learning of embeddings that are both discriminative and topologically structured. We evaluate our method on benchmark datasets (AG News and MBTI) and compare it to classical baselines including TF-IDF, Word2Vec, and unconstrained Keras-derived embeddings. Our results demonstrate that manifold-constrained embeddings, particularly those projected onto spheres and Möbius strips, significantly outperform traditional approaches in both clustering quality (Silhouette Score) and classification performance (Accuracy). These findings highlight the value of embedding in manifold space -- where topological structure complements semantic separation -- offering a new and mathematically grounded direction for geometric representation learning in NLP.
Paper Structure (44 sections, 7 equations, 2 figures, 4 tables)

This paper contains 44 sections, 7 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Sentence Embeddings
  4. Contrastive Learning for Sentence Representations
  5. Manifold-Based Embedding Approaches
  6. Our Contribution
  7. Methodology
  8. Overview of Our Approach
  9. Triplet Loss for Sentence Embedding
  10. Choice of Loss Function.
  11. Manifold Constraints
  12. Sphere:
  13. Torus:
  14. Möbius Strip:
  15. Embedding Network Architecture
  16. ...and 29 more sections

Figures (2)

  • Figure 1: 3D visualizations of AG News sentence embeddings. Manifold embeddings (top) show tighter clusters and cleaner separability compared to Keras baseline (bottom right).
  • Figure 2: 3D embedding visualizations for MBTI dataset. Manifold-constrained embeddings (top and left) exhibit more topological structure than Keras baseline (bottom right).