Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?

TL;DR

This work introduces SymbolBench, a real-world benchmark to assess symbolic reasoning by LLMs over time series across three tasks: multivariate symbolic regression for continuous dynamics (CDEs), Boolean network inference, and causal discovery. It presents a Unified Symbolic Reasoning Framework that orchestrates LLM-based hypothesis generation and evaluation, optionally augmented with genetic programming in a closed-loop loop of refinement. Empirical results show LLMs excel at CDEs and SCMs but lag in BN inference, with context grounding and long-form iterative reasoning (Long CoT) offering modest gains, and LLM-GP hybrids delivering additional improvements. The study highlights the importance of domain-context, structured verification, and hybrid architectures for advancing automated scientific discovery from temporal data.

Abstract

Uncovering hidden symbolic laws from time series data, as an aspiration dating back to Kepler's discovery of planetary motion, remains a core challenge in scientific discovery and artificial intelligence. While Large Language Models show promise in structured reasoning tasks, their ability to infer interpretable, context-aligned symbolic structures from time series data is still underexplored. To systematically evaluate this capability, we introduce SymbolBench, a comprehensive benchmark designed to assess symbolic reasoning over real-world time series across three tasks: multivariate symbolic regression, Boolean network inference, and causal discovery. Unlike prior efforts limited to simple algebraic equations, SymbolBench spans a diverse set of symbolic forms with varying complexity. We further propose a unified framework that integrates LLMs with genetic programming to form a closed-loop symbolic reasoning system, where LLMs act both as predictors and evaluators. Our empirical results reveal key strengths and limitations of current models, highlighting the importance of combining domain knowledge, context alignment, and reasoning structure to improve LLMs in automated scientific discovery.

Figures (15)

• Figure 1: Symbolic structure discovery from time series.
• Figure 2: Iterative refinement framework with hybrid design. Candidate proposals are generated using either an LLM-as-Predictor or genetic programming operations. Each round of candidates undergoes quantitative and qualitative evaluation via validation tools and an LLM-as-Judge. Scored candidates are stored in a history pool, and a context manager decides contextual information for the next round.
• Figure 3: Hybrid method.
• Figure 4: Evaluation scores improve with more iterations and test-time compute, as discussed in Q2.
• Figure 5: Comparison of performance on thinking.