Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Duy Dao Do, Anaïs Halftermeyer, Thi-Bich-Hanh Dao

Abstract

Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Abstract

Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.
Paper Structure (44 sections, 5 equations, 16 figures, 4 tables)

This paper contains 44 sections, 5 equations, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Attention in TRE.
  4. Top-$K$ Attention and Evidence Selection.
  5. Our Positioning.
  6. Preliminaries
  7. Background
  8. Attention Pooling
  9. Methodology
  10. Context Construction
  11. Multi-level Embedding Extraction
  12. Pair-conditioned Top-$K$ Attention
  13. Relation-conditioned cross-attention.
  14. Top-$K$ evidence selection.
  15. Label-aware gating.
  16. ...and 29 more sections

Figures (16)

  • Figure 1: An example from TimeBank-Dense cassidy2014annotation, based on TimeBank 1.2 pustejovsky2006timebank. Explicit signals link BEFORE(takeover, news) and SIMULTANEOUS(spent, thought), while spent-sold depends on implicit contextual cues.
  • Figure 2: Architecture of WISTERIA. BERT and a transformer encoder generate contextualized representations, while pair-conditioned top-$K$ cross-attention extracts key context for each entity pair. The biaffine, context, and pair-fusion outputs are integrated via late fusion for temporal relation classification.
  • Figure 3: Effect of Top-$K$ values on F1 performance across four datasets.
  • Figure 4: Example from the TimeBank-Dense test set illustrating pair-conditioned top-$K$ attention. The model predicts BEFORE between said ($E_1$) and force ($E_2$). Pair-level attention highlights connective cues (e.g., as, itself), while entity-level attention anchors event-specific tokens (e.g., visits, Saddam). The complementary patterns indicate structured evidence selection for temporal inference.
  • Figure 5: POS feature distribution of TimeBank-Dense
  • ...and 11 more figures