Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertion

TL;DR

This work addresses the rigidity of static in-context learning for mathematical reasoning by introducing Process In-Context Learning (PICL), a dynamic framework that detects confusion points during reasoning and retrieves/inserted targeted demonstrations to guide subsequent steps. PICL operates in two stages: confusion point identification using semantics and token entropy, and confusion-based demonstration selection and insertion via a Bi-Encoder for retrieval and a cross-encoder BGEM3-Reranker for re-ranking, with insertion limited by a cap $r$ and per-intervention count $k$. Empirical results across GSM8K, MATH500, AMC23, and AIME24 show PICL consistently outperforms static ICL baselines, achieving up to $83.0\%$ avg accuracy on R1-Qwen-7B and $72.7\%$ on R1-Llama-8B, and giving notable gains on challenging tasks like AIME24. The analysis reveals that staging the intervention around true confusion points and using a small, highly relevant set of demonstrations are key to effectiveness, with entropy-based interruption tokens supporting reliable detection. This dynamic demonstration insertion paradigm advances ICL for step-by-step reasoning and has potential applicability to other complex reasoning domains beyond mathematics.

Abstract

In-context learning (ICL) has proven highly effective across diverse large language model (LLM) tasks. However, its potential for enhancing tasks that demand step-by-step logical deduction, such as mathematical reasoning, remains underexplored. A core limitation of existing ICL approaches is their static use of demonstrations: examples are pre-selected before inference and remain fixed, failing to adapt to the dynamic confusion points that often arise during multi-step reasoning such as ambiguous calculations or logical gaps. These unresolved confusion points can lead to cascading errors that degrade final accuracy. To tackle this issue, we propose Process In-Context Learning (PICL), a dynamic demonstration integration framework designed to boost mathematical reasoning by responding to real-time inference needs. PICL operates in two stages: 1)~it identifies potential confusion points by analyzing semantics and entropy in the reasoning process and summarizes their core characteristics; 2)~upon encountering these points, it retrieves relevant demonstrations from the demonstration pool that match the confusion context and inserts them directly into the ongoing reasoning process to guide subsequent steps. Experiments show that PICL outperforms baseline methods by mitigating mid-inference confusion, highlighting the value of adaptive demonstration insertion in complex mathematical reasoning.

Figures (10)

• Figure 1: Performance of static ICL and the conceptual framework of PICL. (a) Performance comparison of Deepseek-R1-Distilled-Qwen-7B under zero-shot, 1-shot and 4-shot ICL settings on three math benchmarks. (b) A schematic illustration comparing three inference paradigms when encountering a reasoning path selection challenge. Left: Zero-shot prompting. Middle: Conventional static few-shot ICL. Right: Our proposed PICL framework, which inserts targeted demonstrations to provide adaptive guidance.
• Figure 2: Process In-Context Learning (PICL) for enhancing mathematical reasoning of language models. It features two stages: an Interrupt Mechanism detects confusion via semantics and entropy in reasoning. Upon detection, relevant demonstrations are dynamically retrieved from a pool and inserted, refining reasoning process.
• Figure 3: Density distribution of token entropy across reasoning tasks, highlighting high-entropy tokens (e.g., "maybe", "wait") that align with semantic markers of the model's uncertainty and cognitive effort.
• Figure 4: Performance comparison between PICL and Zero-shot with $r$ ranging from $1$ to $4$.
• Figure 5: Performance comparison between PICL and Zero-shot with $k$ ranging from $1$ to $4$.