ConvexBench: Can LLMs Recognize Convex Functions?

TL;DR

This work evaluates whether Large Language Models can identify convexity in deeply composed symbolic functions. It introduces ConvexBench, a scalable benchmark built from Disciplined Convex Programming atoms, with mechanically verifiable labels obtained via DCP rules and Jensen’s inequality checks. The study uncovers a compositional reasoning gap in current LLMs, where performance collapses as depth increases, driven by parsing failures and lazy reasoning. To address this, the authors propose agentic divide-and-conquer frameworks that offload parsing to an external tool and enforce recursive reasoning with focused context, achieving $F1=1.0$ at depth $D=100$ for multiple frontier models and substantially improving robustness on deep compositions.

Abstract

Convex analysis is a modern branch of mathematics with many applications. As Large Language Models (LLMs) start to automate research-level math and sciences, it is important for LLMs to demonstrate the ability to understand and reason with convexity. We introduce \cb, a scalable and mechanically verifiable benchmark for testing \textit{whether LLMs can identify the convexity of a symbolic objective under deep functional composition.} Experiments on frontier LLMs reveal a sharp compositional reasoning gap: performance degrades rapidly with increasing depth, dropping from an F1-score of $1.0$ at depth $2$ to approximately $0.2$ at depth $100$. Inspection of models' reasoning traces indicates two failure modes: \textit{parsing failure} and \textit{lazy reasoning}. To address these limitations, we propose an agentic divide-and-conquer framework that (i) offloads parsing to an external tool to construct an abstract syntax tree (AST) and (ii) enforces recursive reasoning over each intermediate sub-expression with focused context. This framework reliably mitigates deep-composition failures, achieving substantial performance improvement at large depths (e.g., F1-Score $= 1.0$ at depth $100$).

Figures (7)

• Figure 1: F1-Score on $\mathsf{ConvexBench}$ versus composition depth for Qwen3-30B and GPT-5, comparing one-shot reasoning to our agentic reasoning with focused context. One-shot reasoning performance drops from $1.0$ at shallow depth to around $0.2$ at depth $100$, while the agentic framework maintains $1.0$ across depths.
• Figure 2: Overview of $\mathsf{ConvexBench}$ construction. We recursively compose atoms from $\mathcal{A}$ to reach the target depth and convexity, producing an expression with controlled composition depth. For convex/concave targets, outer atoms are chosen to satisfy DCP rules; for neither class, we relax DCP constraints and admit a constructed function only after Jensen's Inequality test finds counterexamples.
• Figure 3: Comparison of different reasoning paradigms on $\mathsf{ConvexBench}$. (1) One-shot Reasoning (Baseline) directly inputs the raw expression into LLMs. (2) One-shot Reasoning with Decomp first decomposes the raw expression into AST, then inputs the AST into LLMs. (3) Agentic Reasoning decomposes the expression into a sequence of sub-tasks and conducts recursive reasoning over each sub-task with full context. (4) Agentic Reasoning with Focused Context constructs a dependency-focused context for each sub-task.
• Figure 4: Evaluation of reasoning performance and consumed tokens across different compositional depths and models. The top row (a-d) shows the F1-Score under different reasoning paradigms. The bottom row (e-h) illustrates the average reasoning tokens consumed.
• Figure 5: Class-wise recall for one-shot reasoning across compositional depth. Convex/concave recall degrades with depth increases, whereas neither recall remains high. Misclassifications increasingly map convex/concave inputs to neither label.