Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces

Yandong Sun, Qiang Huang, Ziwei Xu, Yiqun Sun, Yixuan Tang, Anthony K. H. Tung

TL;DR

This work introduces Semantic Field Subspaces (SFS) as a geometry-preserving, context-aware representation of local semantic neighborhoods within embedding spaces. The unsupervised SAFARI algorithm leverages a novel Semantic Shift metric to uncover hierarchical semantic structures directly from embeddings, without re-embedding or external ontologies. An efficient SVD-based approximation accelerates shift computation, yielding 15–30x speedups with negligible error. Across six real-world datasets spanning text and vision, SFSes improve classification, reveal nuanced semantics (e.g., political bias), and demonstrate consistent, modality-agnostic hierarchical discovery. Together, SFS and SAFARI provide a scalable, interpretable framework for analyzing and leveraging semantic structure in multimodal embeddings.

Abstract

Embedding spaces are fundamental to modern AI, translating raw data into high-dimensional vectors that encode rich semantic relationships. Yet, their internal structures remain opaque, with existing approaches often sacrificing semantic coherence for structural regularity or incurring high computational overhead to improve interpretability. To address these challenges, we introduce the Semantic Field Subspace (SFS), a geometry-preserving, context-aware representation that captures local semantic neighborhoods within the embedding space. We also propose SAFARI (SemAntic Field subspAce deteRmInation), an unsupervised, modality-agnostic algorithm that uncovers hierarchical semantic structures using a novel metric called Semantic Shift, which quantifies how semantics evolve as SFSes evolve. To ensure scalability, we develop an efficient approximation of Semantic Shift that replaces costly SVD computations, achieving a 15~30x speedup with average errors below 0.01. Extensive evaluations across six real-world text and image datasets show that SFSes outperform standard classifiers not only in classification but also in nuanced tasks such as political bias detection, while SAFARI consistently reveals interpretable and generalizable semantic hierarchies. This work presents a unified framework for structuring, analyzing, and scaling semantic understanding in embedding spaces.

One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces

TL;DR

This work introduces Semantic Field Subspaces (SFS) as a geometry-preserving, context-aware representation of local semantic neighborhoods within embedding spaces. The unsupervised SAFARI algorithm leverages a novel Semantic Shift metric to uncover hierarchical semantic structures directly from embeddings, without re-embedding or external ontologies. An efficient SVD-based approximation accelerates shift computation, yielding 15–30x speedups with negligible error. Across six real-world datasets spanning text and vision, SFSes improve classification, reveal nuanced semantics (e.g., political bias), and demonstrate consistent, modality-agnostic hierarchical discovery. Together, SFS and SAFARI provide a scalable, interpretable framework for analyzing and leveraging semantic structure in multimodal embeddings.

Abstract

Embedding spaces are fundamental to modern AI, translating raw data into high-dimensional vectors that encode rich semantic relationships. Yet, their internal structures remain opaque, with existing approaches often sacrificing semantic coherence for structural regularity or incurring high computational overhead to improve interpretability. To address these challenges, we introduce the Semantic Field Subspace (SFS), a geometry-preserving, context-aware representation that captures local semantic neighborhoods within the embedding space. We also propose SAFARI (SemAntic Field subspAce deteRmInation), an unsupervised, modality-agnostic algorithm that uncovers hierarchical semantic structures using a novel metric called Semantic Shift, which quantifies how semantics evolve as SFSes evolve. To ensure scalability, we develop an efficient approximation of Semantic Shift that replaces costly SVD computations, achieving a 15~30x speedup with average errors below 0.01. Extensive evaluations across six real-world text and image datasets show that SFSes outperform standard classifiers not only in classification but also in nuanced tasks such as political bias detection, while SAFARI consistently reveals interpretable and generalizable semantic hierarchies. This work presents a unified framework for structuring, analyzing, and scaling semantic understanding in embedding spaces.

Paper Structure

This paper contains 65 sections, 4 theorems, 5 equations, 14 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Structural Analysis of Embedding Spaces
  4. Semantic and Hierarchical Structure Discovery
  5. Problem Formulation
  6. Vector Space Foundation
  7. Challenge of Context-Dependent Meaning
  8. Semantic Fields in Embedding Spaces
  9. Research Objective: From Local to Global Semantics
  10. Methodology
  11. Semantic Field Representation
  12. Geometric Representation: Semantic Field Subspace (SFS)
  13. SFS Boundary Delineation via Semantic Shift
  14. The SAFARI Algorithm
  15. Algorithm Overview
  16. ...and 50 more sections

Key Result

Proposition 3.1

An embedding vector cannot be semantically interpreted in isolation.

Figures (14)

  • Figure 1: Contextual interpretation of Apple: Meaning refines as more related terms are introduced.
  • Figure 2: Illustration of Semantic Field exploration.
  • Figure 3: Toy example illustrating SAFARI's hierarchical clustering process.
  • Figure 4: Impurity across hierarchical levels.
  • Figure 5: Runtime comparison between exact and approximate Semantic Shift computation across seven topic classes.
  • ...and 9 more figures

Theorems & Definitions (10)

  • Definition 3.1: Semantic Distance
  • Proposition 3.1: Context-Dependent Meaning
  • Example 3.1
  • Definition 3.2: Semantic Field
  • Definition 4.1: Semantic Field Subspace (SFS)
  • Proposition 4.1: Hierarchical Semantic Structure
  • Definition 4.2: Semantic Shift
  • Example 4.1
  • Theorem 4.1: Bound on Dimensional Importance Shift
  • Theorem 4.2: Weyl's Theorem weyl1912asymptotische