Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams

Abstract

Evaluating language models in streaming environments is critical, yet underexplored. Existing benchmarks either focus on single complex events or provide curated inputs for each query, and do not evaluate models under the conflicts that arise when multiple concurrent events are mixed within the same document stream. We introduce StreamBench, a benchmark built from major news stories in 2016 and 2025, comprising 605 events and 15,354 documents across three tasks: Topic Clustering, Temporal Question Answering, and Summarization. To diagnose how models fail, we compare performance with and without structural cues, which organize key facts by event. We find that structural cues improve performance on clustering (up to +4.37%) and temporal QA (up to +9.63%), helping models locate relevant information and separate distinct events. While temporal reasoning remains an open challenge inherent to current LLMs, consistent gains across tasks show that structural cues are a promising direction for future work in massive document streams.

Figures (10)

• Figure 1: Two challenges in streaming environments. Intra-topic conflict: within Dixie Fire, the most recent event (Event 3) has fewer documents than the outdated event (Event 1), making it harder to find the latest information. Inter-topic conflict: when asked about Dixie Fire (Topic 1), the LLM may confuse "8 firefighters" (Event 2) from Bootleg Fire (Topic 2).
• Figure 2: Document stream volume over time. The x-axis shows the normalized story timeline, and the y-axis indicates the number of documents per 7-day window. We vary $k \in {1, 3, 5, 10}$, the number of documents sampled per event. The 2016 stories (D–F) contain more documents per window than the 2025 stories (A–C).
• Figure 3: Performance bottleneck analysis across tasks and model scales. Stacked bar charts show base performance and structure cue effects for Small (1--4B) and Large (70B+) models. Green bar indicate positive effect from structural cues; red indicates negative effect. Hatched bar represent headroom to the ceiling. $k$ indicate number of documents sampled per event.
• Figure 4: $\Delta_{\text{org}}$ across model scales and document sizes per event ($k$) for each task. * indicates statistical significance ($p < 0.05$).
• Figure 5: CheckEval results for summarization, with $\Delta_{\text{org}}$ broken down by evaluation dimension.