The Imperfective Paradox in Large Language Models

TL;DR

This work investigates whether large language models truly understand the compositional semantics of events or rely on surface heuristics. By introducing ImperfectiveNLI, a diagnostic dataset that contrasts telic and atelic verbs under interrupted and ambiguous contexts, the authors reveal a pervasive Teleological Bias in open-weight LLMs, where progressives often imply completion for goal-directed actions. Prompting strategies can mitigate this bias but induce calibration costs, and the bias arises at inference time despite separable internal representations of process versus result, suggesting a need for Aspect-Aware Alignment rather than purely prompting-based fixes. The results emphasize non-linear emergent aspectual reasoning with model scale and highlight the importance of fine-grained verb semantics, underscoring the gap between linguistic aspectual theory and current neural reasoning capabilities. The work advocates building structurally grounded aspectual reasoning into LLMs to achieve faithful event semantics beyond predictive narrative capabilities.

Abstract

Do Large Language Models (LLMs) genuinely grasp the compositional semantics of events, or do they rely on surface-level probabilistic heuristics? We investigate the Imperfective Paradox, a logical phenomenon where the past progressive aspect entails event realization for activities (e.g., running $\to$ ran) but not for accomplishments (e.g., building $\nrightarrow$ built). We introduce ImperfectiveNLI, a diagnostic dataset designed to probe this distinction across diverse semantic classes. Evaluating state-of-the-art open-weight models, we uncover a pervasive Teleological Bias: models systematically hallucinate completion for goal-oriented events, often overriding explicit textual negation. Representational analyses show that while internal embeddings often distinguish process from result, inference decisions are dominated by strong priors about goal attainment. We further find that prompting-based interventions reduce hallucinated completions but also increase incorrect rejections of valid entailments. Our findings suggest that current LLMs lack structural aspectual awareness, operating as predictive narrative engines rather than faithful logical reasoners.

Figures (9)

• Figure 1: The trade-off in prompting (Llama-3.1). As prompts become more aggressive in targeting bias (Zero-shot $\to$ Counterfactual), accuracy on Group C improves significantly (Pearson $r=1.00, p=0.00$), and accuracy on Group D collapses also, only unsteadily ($r=-0.91, p=0.09$). This demonstrates the calibration bias.
• Figure 2: The emergence of aspectual reasoning capabilities across model scales. It shows a significant decrease in TBR and a corresponding increase in $\Delta_{AA}$.
• Figure 3: Average accuracy across models by semantic verb classes in Zero-shot. Motion verbs are consistently easier for models to reason about, while Creation verbs induce the stronge teleological bias (lowest accuracy in Group C) and resistance to contextual cancellation (lowest accuracy in Group A).
• Figure 4: Representational vs. Behavioral Divergence.
• Figure 5: The Trade-off in Prompting (Mistral). The model shows a clear skepticism trap with strong correlations. (Group C: $r=0.95, p=0.05$; Group D: $r=-0.93, p=0.07$).