ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following

TL;DR

This work proposes ImpRIF, a method to enhance LLMs' understanding of implicit reasoning instructions, thereby improving its ability to follow complex instructions, and demonstrates that enhancing implicit reasoning capabilities can significantly improve complex instruction following.

Abstract

As applications of large language models (LLMs) become increasingly complex, the demand for robust complex instruction following capabilities is growing accordingly. We argue that a thorough understanding of the instruction itself, especially the latent reasoning structure embedded between the lines, is crucial for improving instruction following. Therefore we target complex instructions that involve implicit reasoning, intricate logical relations, and multi-constraint dependencies. We propose ImpRIF, a method to enhance LLMs' understanding of implicit reasoning instructions, thereby improving its ability to follow complex instructions. We formalize such instructions as verifiable reasoning graphs, enabling programmatic verification and graph-driven chain-of-thought reasoning. Based on this formulation, we synthesize large-scale single- and multi-turn data, propose fine-tuning with graph reasoning, and apply reinforcement learning to explicitly train models to reason along the graph. On five complex instruction following benchmarks, our models substantially outperform their base models. These results demonstrate that enhancing implicit reasoning capabilities can significantly improve complex instruction following. This project will be open-sourced in the near future.

Figures (7)

• Figure 1: Comparison of complex instruction structures. The top illustrates simple combinations of multiple constraints in traditional complex instructions, while the bottom shows the node dependency graph of our proposed implicit multi-hop reasoning instructions, highlighting the implicit reasoning structures we target.
• Figure 2: Overview of the proposed pipeline. The top depicts the generation process of implicit reasoning data, the middle provides a detailed breakdown of constraint generation and instruction construction, and the bottom shows SFT with ERG CoT and RL training with process verification.
• Figure 3: The performance of frontier models and our trained ImpRIF-32B on our internal test set and open benchmarks. GPT-4.1 lacks reasoning capability; all other models have reasoning enabled.
• Figure 4: The performance of our model on LogicBench. All trained versions outperform the base model.
• Figure 5: Reward curves during RL training. We compare reward trajectories over training steps for ImpRIF-4B (left) and ImpRIF-8B (right) under the RL-only and SFT$\rightarrow$RL pipelines.