NPG-Muse: Scaling Long Chain-of-Thought Reasoning with NP-Hard Graph Problems

TL;DR

This work tackles the scalability of long chain-of-thought reasoning in large language models by proposing NP-hard graph problems as a scalable synthetic training corpus. It introduces a two-stage post-training pipeline—Long-CoT supervised fine-tuning on rejection-sampled NP-hard graph instances followed by reinforcement learning with a fine-grained reward design—producing the NPG-Muse-series. The resulting models show substantial improvements in Long CoT capabilities and cross-domain generalization, outperforming larger baselines on NP-hard graph problems and achieving strong transfers to mathematics, coding, logic, and graph reasoning tasks. The approach demonstrates a scalable path to enhance deep, exploratory, and reflective reasoning in LLMs, with practical implications for complex problem solving and reasoning benchmarks.

Abstract

Reasoning Large Language Models (RLLMs) have recently achieved remarkable progress on complex reasoning tasks, largely enabled by their long chain-of-thought (Long CoT) capabilities. However, developing these Long CoT behaviors relies heavily on post-training with high-quality datasets, which are typically costly and human-curated (e.g., mathematics and code), leaving scalable alternatives unexplored. In this work, we introduce NP-hard (NPH) graph problems as a novel synthetic training corpus, as they inherently require deep reasoning, extensive exploration, and reflective strategies, which are the core characteristics of Long CoT reasoning. Building on this insight, we develop a two-stage post-training framework: (i) Long-CoT Supervised Fine-Tuning (SFT) on rejection-sampled NPH graph instances, which substantially enhances reasoning depth, and (ii) Reinforcement Learning (RL) with a fine-grained reward design, which sharpens reasoning efficiency. The resulting NPG-Muse-series models exhibit substantially enhanced Long CoT reasoning capabilities, achieving consistent gains across mathematics, coding, logical, and graph reasoning benchmarks. NPG-Muse-7B even surpasses QwQ-32B on NPH graph problems in both accuracy and reasoning efficiency. These results position NPH graph problems as an effective and scalable resource for advancing Long CoT reasoning in LLM post-training. Our implementation is available at https://github.com/littlewyy/NPG-Muse.

Figures (6)

• Figure 1: Overview of NPG-Muse.
• Figure 2: Methodology of NPG-Muse.
• Figure 3: Token efficiency compared to QwQ-32B.
• Figure 4: Self-reflection behavior statistics.
• Figure 5: Case study for TSP task.