Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Is More Context Always Better? Examining LLM Reasoning Capability for Time Interval Prediction

Yanan Cao, Farnaz Fallahi, Murali Mohana Krishna Dandu, Lalitesh Morishetti, Kai Zhao, Luyi Ma, Sinduja Subramaniam, Jianpeng Xu, Evren Korpeoglu, Kaushiki Nag, Sushant Kumar, Kannan Achan

TL;DR

This study probes whether LLMs can infer discrete time intervals between user actions, such as inter-purchase gaps, using zero-shot prompts with varying context. By benchmarking GPT-4o, Claude 3.5, and Gemini against statistical and ML baselines on two real-world datasets, it shows LLMs surpass simple statistics but underperform dedicated ML models in exact interval prediction, highlighting an impedance mismatch between linguistic reasoning and quantitative forecasting. A key finding is that moderate context improves LLM accuracy, while richer high-context narratives can degrade performance, indicating context can act as noise for temporal inference. The work emphasizes the need for hybrid models that blend statistical precision with linguistic flexibility and provides a rigorous framework for evaluating time-interval prediction with structured data.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning and prediction across different domains. Yet, their ability to infer temporal regularities from structured behavioral data remains underexplored. This paper presents a systematic study investigating whether LLMs can predict time intervals between recurring user actions, such as repeated purchases, and how different levels of contextual information shape their predictive behavior. Using a simple but representative repurchase scenario, we benchmark state-of-the-art LLMs in zero-shot settings against both statistical and machine-learning models. Two key findings emerge. First, while LLMs surpass lightweight statistical baselines, they consistently underperform dedicated machine-learning models, showing their limited ability to capture quantitative temporal structure. Second, although moderate context can improve LLM accuracy, adding further user-level detail degrades performance. These results challenge the assumption that "more context leads to better reasoning". Our study highlights fundamental limitations of today's LLMs in structured temporal inference and offers guidance for designing future context-aware hybrid models that integrate statistical precision with linguistic flexibility.

Is More Context Always Better? Examining LLM Reasoning Capability for Time Interval Prediction

TL;DR

This study probes whether LLMs can infer discrete time intervals between user actions, such as inter-purchase gaps, using zero-shot prompts with varying context. By benchmarking GPT-4o, Claude 3.5, and Gemini against statistical and ML baselines on two real-world datasets, it shows LLMs surpass simple statistics but underperform dedicated ML models in exact interval prediction, highlighting an impedance mismatch between linguistic reasoning and quantitative forecasting. A key finding is that moderate context improves LLM accuracy, while richer high-context narratives can degrade performance, indicating context can act as noise for temporal inference. The work emphasizes the need for hybrid models that blend statistical precision with linguistic flexibility and provides a rigorous framework for evaluating time-interval prediction with structured data.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning and prediction across different domains. Yet, their ability to infer temporal regularities from structured behavioral data remains underexplored. This paper presents a systematic study investigating whether LLMs can predict time intervals between recurring user actions, such as repeated purchases, and how different levels of contextual information shape their predictive behavior. Using a simple but representative repurchase scenario, we benchmark state-of-the-art LLMs in zero-shot settings against both statistical and machine-learning models. Two key findings emerge. First, while LLMs surpass lightweight statistical baselines, they consistently underperform dedicated machine-learning models, showing their limited ability to capture quantitative temporal structure. Second, although moderate context can improve LLM accuracy, adding further user-level detail degrades performance. These results challenge the assumption that "more context leads to better reasoning". Our study highlights fundamental limitations of today's LLMs in structured temporal inference and offers guidance for designing future context-aware hybrid models that integrate statistical precision with linguistic flexibility.
Paper Structure (6 sections, 3 figures, 1 table)

This paper contains 6 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Experimental Setting
  3. Datasets
  4. Models
  5. Experimental Results
  6. Main Results

Figures (3)

  • Figure 1: Illustration of the interval-prediction task and the three prompting conditions. The example is about repeated milk purchases with varying intervals (5 days → 9 days → 7 days → 6 days). The model observes historical intervals for a product category and predicts the next interval under zero, medium, and high context settings.
  • Figure 2: Prompt designs for three context levels: Zero (historical intervals only), Medium (product metadata, summary statistics), and High (recency features, user lifecycle information).
  • Figure 3: Examples of Claude-3.5-Sonnet prediction and reasoning with different levels of context. Green marks the correct medium level information leveraged, while red marks the additional high context leading to incorrect prediction.