Extracting Event Temporal Relations via Hyperbolic Geometry

TL;DR

This work tackles the challenge of extracting and reasoning about temporal relations between events by moving beyond Euclidean embeddings to hyperbolic geometry, which naturally captures hierarchical and asymmetric structures. It proposes two hyperbolic strategies: (i) a Poincaré-ball embedding method with an angular loss and a simple rule-based classifier, and (ii) an end-to-end hyperbolic neural network (HGRU) that processes contextualized text through hyperbolic layers to predict relation types, optionally leveraging TemProb commonsense. The methods achieve state-of-the-art results on MATRES and TCR, with ablations showing the hyperbolic components—distance, angular constraints, and hyperbolic recurrence—drive performance gains, and qualitative analyses demonstrating richer event semantics encoded in hyperbolic space. The findings highlight the viability and benefits of non-Euclidean geometry for temporal reasoning in NLP and suggest directions for extending to causal and sub-event relations.

Abstract

Detecting events and their evolution through time is a crucial task in natural language understanding. Recent neural approaches to event temporal relation extraction typically map events to embeddings in the Euclidean space and train a classifier to detect temporal relations between event pairs. However, embeddings in the Euclidean space cannot capture richer asymmetric relations such as event temporal relations. We thus propose to embed events into hyperbolic spaces, which are intrinsically oriented at modeling hierarchical structures. We introduce two approaches to encode events and their temporal relations in hyperbolic spaces. One approach leverages hyperbolic embeddings to directly infer event relations through simple geometrical operations. In the second one, we devise an end-to-end architecture composed of hyperbolic neural units tailored for the temporal relation extraction task. Thorough experimental assessments on widely used datasets have shown the benefits of revisiting the tasks on a different geometrical space, resulting in state-of-the-art performance on several standard metrics. Finally, the ablation study and several qualitative analyses highlighted the rich event semantics implicitly encoded into hyperbolic spaces.

Figures (4)

• Figure 1: Events annotated with temporal relations from a document excerpt. Arrow lines represent the Before relations, while red dashed lines the Vague ones.
• Figure 2: An illustration of the Poincaré embedding used to encode two events $u$ and $v$ with known temporal relation. $\theta_1$ is the angle between the event pair $(u,v)$, while $\theta_2$ is the angle of an event pair $(u, v')$ resulting from the negative sampling process.
• Figure 3: In the hyperbolic neural architecture for temporal relation extraction, sentence tokens are first associated with standard RoBERTa vectors (within the Euclidean space). They are subsequently mapped into a Poincaré ball and processed using Hyperbolic Feed-Forward Layers (H-FFL) and Hyperbolic-GRUs (H-GRU). Then, a masking process ensures that only the event-related vectors are aggregated via Möbius operations, along with their $d_\mathbb{D}$ distance and the relevant temporal common sense, extracted by a Siamese network pre-trained on TemProb knowledge base. Finally, the distribution over event temporal relations is derived using a Hyperbolic Multinomial Logistic Regression (H-MLR), analogous to a traditional Softmax layer in the Euclidean space.
• Figure 4: A document excerpt from the MATRES dataset and the related temporal event embedding generated by the Poincaré embedding method.