NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problems

TL;DR

This work tackles the challenge of optimization reasoning on NP-hard problems by introducing NP-Engine, a scalable RLVR framework that combines generator–verifier–heuristic pipelines to train LLMs on 10 NP-hard tasks. It also defines NP-Bench, a 1000-instance-level benchmark designed to evaluate both feasibility and solution quality via SR and AR, grounded by task-specific heuristics. The authors train Qwen2.5-7B-NP with zero-shot RLVR and curriculum learning, achieving state-of-the-art in-domain performance and notable out-of-domain generalization, outperforming GPT-4o on NP-Bench. Key findings include that increasing task diversity and employing multi-stage, curriculum-guided RLVR significantly improve generalization and reveal scaling trends for complex reasoning in NP-type tasks, suggesting a practical path to enhanced optimization reasoning in LLMs.

Abstract

Large Language Models (LLMs) have shown strong reasoning capabilities, with models like OpenAI's O-series and DeepSeek R1 excelling at tasks such as mathematics, coding, logic, and puzzles through Reinforcement Learning with Verifiable Rewards (RLVR). However, their ability to solve more complex optimization problems - particularly NP-hard tasks - remains underexplored. To bridge this gap, we propose NP-ENGINE, the first comprehensive framework for training and evaluating LLMs on NP-hard problems. NP-ENGINE covers 10 tasks across five domains, each equipped with (i) a controllable instance generator, (ii) a rule-based verifier, and (iii) a heuristic solver that provides approximate optimal solutions as ground truth. This generator-verifier-heuristic pipeline enables scalable and verifiable RLVR training under hierarchical difficulties. We also introduce NP-BENCH, a benchmark derived from NP-ENGINE-DATA, specifically designed to evaluate LLMs' ability to tackle NP-hard level reasoning problems, focusing not only on feasibility but also on solution quality. Additionally, we present QWEN2.5-7B-NP, a model trained via zero-RLVR with curriculum learning on Qwen2.5-7B-Instruct, which significantly outperforms GPT-4o on NP-BENCH and achieves SOTA performance with the same model size. Beyond in-domain tasks, we demonstrate that RLVR training on NP-ENGINE-DATA enables strong out-of-domain (OOD) generalization to reasoning tasks (logic, puzzles, math, and knowledge), as well as non-reasoning tasks such as instruction following. We also observe a scaling trend: increasing task diversity improves OOD generalization. These findings suggest that task-rich RLVR training is a promising direction for advancing LLM's reasoning ability, revealing new insights into the scaling laws of RLVR.

Figures (3)

• Figure 1: Overview of the NP-Engine-Data: 10 NP-hard level tasks across five categories (graph clustering, resource scheduling, graph partitioning, subset selection, and path planning), designed to improve and evaluate optimization reasoning capabilities in LLMs.
• Figure 2: Overview of NP-Engine. The suite comprises ten NP-hard tasks spanning five categories (graph structure, scheduling, partitioning, subset selection, and tour planning). Each task is equipped with scalable instance generators and verifiers and is stratified into Easy/Medium/Hard, enabling RLVR curriculum training in LLMs.
• Figure 3: Comparison of RL training strategies during multi-task training, with performance evaluated on both in-domain and out-of-domain benchmarks.