Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces

Hossein Javidnia

Abstract

Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.

Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces

Abstract

Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.
Paper Structure (44 sections, 3 theorems, 18 equations, 2 figures, 9 tables)

This paper contains 44 sections, 3 theorems, 18 equations, 2 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Mechanistic interpretability and global feature ontologies.
  5. Superposition, sparse decompositions, and monosemantic features.
  6. Geometric and local-to-global views of representation.
  7. What is new here.
  8. Semantic sections and atlas-native features
  9. Semantic sections
  10. Tree-local extension
  11. Cycle-supported globalization
  12. Holonomy obstruction
  13. Twisted sections
  14. Discovery and certification of semantic sections
  15. Seeded propagation on the chart graph
  16. ...and 29 more sections

Key Result

Proposition 3.2

Let $H=(V_H,E_H)$ be a connected tree, let $r\in V_H$ be a root, and let $s_r\in F_r\setminus\{0\}$ be a seed representative. Assume that along every edge of the rooted tree, the transported representative is nonzero. Then there exists a normalized section $s=\{s_c\}_{c\in V_H}$, unique up to an ove In particular, every seed extends uniquely along the tree by recursive transport and normalization.

Figures (2)

  • Figure 1: Semantic sections on a cyclic chart subsystem. (A) A globalizable section, where transporting the local representative $s_u$ around a loop $\gamma$ returns a representative $T_\gamma s_u$ that remains nearly aligned with $s_u$, yielding small loop defect $\Delta_\gamma \approx 0$. (B) A twisted section, where the same loop transport returns a rotated representative, so $T_\gamma s_u$ no longer aligns with $s_u$ and the loop defect is nonzero, $\Delta_\gamma > 0$. This illustrates the difference between cycle-consistent semantic globalization and holonomy-obstructed local coherence.
  • Figure 2: Aggregate semantic identity recovery across section types. Section-based recovery remains perfect on certified supports, while raw global-similarity recovery is consistently lower and typically collapses at stricter thresholds.

Theorems & Definitions (11)

  • Definition 3.1: Semantic section
  • Proposition 3.2: Tree extension
  • proof
  • Remark 3.3
  • Definition 3.4: Globalizable section
  • Theorem 3.5: Cycle consistency criterion
  • proof : Proof sketch
  • Corollary 3.6: Holonomy obstructs single-valued semantic globalization
  • proof
  • Definition 3.7: Twisted section
  • ...and 1 more