$A^3$-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activation

TL;DR

A^3-Bench introduces a memory-driven benchmark for scientific reasoning grounded in Anchor and Attractor Activation, addressing gaps in existing benchmarks that miss memory activation dynamics. It annotates 2,198 problems via the SAPM process, develops a dual-scale memory framework, and proposes the AAUI metric to quantify memory activation during reasoning. Across ten LLMs and multiple paradigms, memory-augmented reasoning consistently improves accuracy, especially on hard problems, and AAUI correlates with reasoning fidelity. The work provides a cognitively aligned, interpretable evaluation that generalizes beyond the source data and offers actionable signals to guide memory-driven model improvements.

Abstract

Scientific reasoning relies not only on logical inference but also on activating prior knowledge and experiential structures. Memory can efficiently reuse knowledge and enhance reasoning consistency and stability. However, existing benchmarks mainly evaluate final answers or step-by-step coherence, overlooking the \textit{memory-driven} mechanisms that underlie human reasoning, which involves activating anchors and attractors, then integrating them into multi-step inference. To address this gap, we propose $A^3$-Bench~ https://a3-bench.github.io, a benchmark designed to evaluate scientific reasoning through dual-scale memory-driven activation, grounded in Anchor and Attractor Activation. First, we annotate 2,198 science reasoning problems across domains using the SAPM process(subject, anchor & attractor, problem, and memory developing). Second, we introduce a dual-scale memory evaluation framework utilizing anchors and attractors, along with the AAUI(Anchor--Attractor Utilization Index) metric to measure memory activation rates. Finally, through experiments with various base models and paradigms, we validate $A^3$-Bench and analyze how memory activation impacts reasoning performance, providing insights into memory-driven scientific reasoning.

Figures (10)

• Figure 1: Comparison of reasoning paths on OlympiadBench. Activating anchors and attractors corrects the derivation path relative to no memory.
• Figure 2: Performance and token analysis across ten LLMs and three memory paradigms. The three color-coded groups represent the experimental paradigms: vanilla, anchors & attractors, and annotated anchors & attractors.
• Figure 3: The four-step annotation process SAPM. First, subject benchmarking defines subdomains for each discipline. Next, experts develop anchors and attractors for each subdomain and define their relations. Then, a new set of questions is refined from existing datasets. Finally, memory mapping associates questions with relevant anchors and attractors.
• Figure 4: A piece of math problem in $A^3$-Bench.
• Figure 5: Schema of the $A^3$-Bench dataset and its usage within a HybridRAG framework. (a) memory twin-needle activator. (b) context fabric composer.