Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?

Gagan Bhatia, Ahmad Muhammad Isa, Maxime Peyrard, Wei Zhao

Abstract

We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains $15,000$ examples built by translating $750$ curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb

What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?

Abstract

We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains examples built by translating curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb
Paper Structure (52 sections, 5 equations, 8 figures, 10 tables)

This paper contains 52 sections, 5 equations, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Tokenisation bias in multilingual models.
  4. Mechanisms of time: memorisation vs. reasoning.
  5. Our MultiTempBench
  6. Dataset construction.
  7. Multilingual extension.
  8. Data format extension.
  9. Our Approach
  10. Multilingual Date Fragmentation Ratio
  11. Temporal Geometry
  12. Embedding extraction.
  13. Geometric notations.
  14. Linear structure of time.
  15. Experiments
  16. ...and 37 more sections

Figures (8)

  • Figure 1: Mechanistic understanding of multilingual temporal reasoning in MultiTempBench.
  • Figure 2: Impact of Tokenisation on Date Arithmetic Accuracy. DFR is strongly negatively correlated with accuracy in Hausa ($r=-0.97$), but only weakly correlated in English ($r=-0.17$).
  • Figure 3: Temporal linearity vs. accuracy across languages. Temporal linearity (probe $R^2$) is strongly correlated with accuracy in English ($r{=}0.77$) and Chinese ($r{=}0.75$), but weakly correlated in Hausa ($r{=}0.10$), suggesting that ordered temporal geometry is a key driver of high performance when it emerges.
  • Figure 4: Component-wise temporal linearity vs. accuracy. Correlations between accuracy and probe $R^2$ for Day, Month, and Year within each language.
  • Figure 5: Mixed-effects summary of temporal reasoning bottlenecks. (a) Fixed effects from the crossed mixed-effects regression. (b) Dominant predictor by resource regime: mDFR in low-resource languages, linearity in high-resource languages.
  • ...and 3 more figures