Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences

Antonin Sulc

TL;DR

Event2Vec introduces a geometry-guided framework for learning composable representations of discrete event sequences. By enforcing an additive structure via a reconstruction loss and grounding semantics through a prediction objective, the Euclidean variant provably approaches an ideal additive form, while a hyperbolic variant using Möbius addition models hierarchical, tree-like data. Empirically, the approach yields interpretable trajectories and improves syntactic structure capture over Word2Vec on the Brown Corpus, with qualitative Life Path analyses highlighting analogical reasoning capabilities. The work demonstrates how simple geometric priors can yield mechanistically interpretable embeddings and motivates hybrid models that balance additive interpretability with richer dynamics.

Abstract

The study of neural representations, both in biological and artificial systems, is increasingly revealing the importance of geometric and topological structures. Inspired by this, we introduce Event2Vec, a novel framework for learning representations of discrete event sequences. Our model leverages a simple, additive recurrent structure to learn composable, interpretable embeddings. We provide a theoretical analysis demonstrating that, under specific training objectives, our model's learned representations in a Euclidean space converge to an ideal additive structure. This ensures that the representation of a sequence is the vector sum of its constituent events, a property we term the linear additive hypothesis. To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space, which is naturally suited to embedding tree-like structures with low distortion. We present experiments to validate our hypothesis. Quantitative evaluation on the Brown Corpus yields a Silhouette score of 0.0564, outperforming a Word2Vec baseline (0.0215), demonstrating the model's ability to capture structural dependencies without supervision.

Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences

TL;DR

Event2Vec introduces a geometry-guided framework for learning composable representations of discrete event sequences. By enforcing an additive structure via a reconstruction loss and grounding semantics through a prediction objective, the Euclidean variant provably approaches an ideal additive form, while a hyperbolic variant using Möbius addition models hierarchical, tree-like data. Empirically, the approach yields interpretable trajectories and improves syntactic structure capture over Word2Vec on the Brown Corpus, with qualitative Life Path analyses highlighting analogical reasoning capabilities. The work demonstrates how simple geometric priors can yield mechanistically interpretable embeddings and motivates hybrid models that balance additive interpretability with richer dynamics.

Abstract

The study of neural representations, both in biological and artificial systems, is increasingly revealing the importance of geometric and topological structures. Inspired by this, we introduce Event2Vec, a novel framework for learning representations of discrete event sequences. Our model leverages a simple, additive recurrent structure to learn composable, interpretable embeddings. We provide a theoretical analysis demonstrating that, under specific training objectives, our model's learned representations in a Euclidean space converge to an ideal additive structure. This ensures that the representation of a sequence is the vector sum of its constituent events, a property we term the linear additive hypothesis. To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space, which is naturally suited to embedding tree-like structures with low distortion. We present experiments to validate our hypothesis. Quantitative evaluation on the Brown Corpus yields a Silhouette score of 0.0564, outperforming a Word2Vec baseline (0.0215), demonstrating the model's ability to capture structural dependencies without supervision.

Paper Structure

This paper contains 15 sections, 3 theorems, 8 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Event2Vec Model
  4. Model Architecture
  5. Euclidean Model
  6. Hyperbolic Model
  7. Training Objective
  8. Theoretical Justification for Additivity
  9. Experiments
  10. Life Path Example
  11. Geometric Analysis of Embedding
  12. Comparison of Euclidean vs. Hyperbolic Space:
  13. Brown Corpus
  14. Limitations
  15. Conclusion

Key Result

theorem 1

Let the hidden state update be defined as $h_t = f(h_{t-1}, e_{s_t})$, where $f$ is a function parameterized by the neural network. Minimizing the reconstruction loss $\mathcal{L}_{recon} = \sum_{t} \| f(h_t, -e_{s_t}) - h_{t-1} \|^2_2$ with respect to the parameters of $f$ drives $f$ to approximate

Figures (12)

  • Figure 1: Geometric Properties of Event2Vec Embeddings. The model learns to represent event sequences as trajectories in a vector space. (A) Probable event sequences form trajectories where consecutive event vectors are directionally aligned. For example, two distinct life paths ('homeschool' $\rightarrow$ 'career' and 'startup' $\rightarrow$ 'career') converge towards a similar state ('death') by adding event vectors that follow a logical progression. (B) The additive structure enables analogical reasoning through vector arithmetic. The "formalization" vector learned from the 'engagement' to 'marriage' transition can be applied to 'parenthood' to correctly identify the parallel concept of 'adoption'. (C) Euclidean Space: Unlike context-based models which cluster by semantic similarity, Event2Vec arranges events by sequential proximity. Distant points on the plot represent distant points in time, preserving the chronological trajectory. (D) Hyperbolic Space: With its exponential volume, it is better suited for capturing hierarchical branching, preventing the 'crowding' of distinct paths.
  • Figure 2: A t-SNE comparison of life path embeddings. (Left) The Euclidean Event2Vec model learns a clear chronological trajectory. (Center) The Hyperbolic Event2Vec model captures a more powerful hierarchical structure, with life paths branching radially from the 'birth' event. (Right) The Word2Vec baseline learns thematic clusters, grouping events by semantic context (e.g., 'birth' and 'death') while failing to capture sequential or hierarchical relationships.
  • Figure 3: t-SNE visualization of embedded POS tag sequences from the Brown Corpus. Event2Vec (left) produces well-defined clusters corresponding to grammatical structures, correctly grouping similar sequences like 'AT-JJ-NN' and 'IN-AT-NN'. The Word2Vec baseline (right) shows significantly weaker separation and overlapping clusters.
  • Figure 4: A high-level overview of the life model, where each node represents a major life stage. The directed edges indicate the flow between stages, with the label and thickness of each edge representing the total number of distinct transition paths. This illustrates the central roles of the Career and Health & Finances stages as highly interconnected hubs in a person's life journey.
  • Figure 5: The Early Life stage, showing the foundational years from the 'birth' event through childhood. The diagram shows a strong, linear progression through key developmental milestones, from 'infancy' to 'Late Childhood', with high transition probabilities indicating a canonical path. The stage concludes with transitions to later life phases such as 'Middle School' and the formation of 'Friendship'. Notably, the model also captures the persistent, low-probability risk of a 'Health Issue' occurring at any point during this period, demonstrating its ability to model both sequential and parallel events.
  • ...and 7 more figures

Theorems & Definitions (6)

  • theorem 1: Justification for Ideal Additivity
  • proof
  • theorem 2: Semantic Grounding via Prediction Loss
  • proof : Sketch
  • theorem 3: Hyperbolic Equivalence
  • proof