Eidoku: A Neuro-Symbolic Verification Gate for LLM Reasoning via Structural Constraint Satisfaction

TL;DR

This work reframes verification of LLM reasoning as a feasibility problem over semantic structure rather than a likelihood-based confidence task. It introduces Eidoku, a minimal System-2 gate that computes a total semantic Violation Cost $J(S)$ as an additive sum of structural, geometric, and logical proxies, and uses a context-adaptive threshold $\tau_c$ to reject structurally unsupported steps. The approach demonstrates that enforcing structural closure can deterministically reject smooth falsehoods on a synthetic Reasoning Gap Dataset (RGD) where high-probability falsehoods are structurally disconnected, achieving zero FTAR with perfect TTAR. By decoupling verification from generation and calibrating the feasibility boundary per context, this method offers an interpretable, scalable mechanism for improving the safety of generative reasoning in real-world deployments.

Abstract

Large Language Models (LLMs) frequently produce hallucinated statements that are assigned high likelihood by the model itself, exposing a fundamental limitation of probability-based verification. This suggests that hallucination is often not a low-confidence phenomenon, but a failure of structural consistency. In this work, we reformulate the verification of LLM reasoning as a Constraint Satisfaction Problem (CSP) operating independently of the generation likelihood. Rather than optimizing for statistical plausibility, we model verification as a feasibility check based on structural violation cost -- the computational cost required to embed a candidate reasoning step into the contextual graph structure. We define a total cost function composed of three proxies: (i) graph connectivity (structural), (ii) feature space consistency (geometric), and (iii) logical entailment (symbolic). Crucially, verification is performed via a lightweight System-2 gate, Eidoku, which rejects candidates exceeding a context-calibrated cost threshold. The threshold is not learned but is derived from the intrinsic statistics of the context, avoiding ad hoc heuristics. We demonstrate that this approach successfully rejects ``smooth falsehoods'' -- statements that are highly probable yet structurally disconnected -- that probability-based verifiers are principally incapable of detecting. Our experiments on a controlled diagnostic dataset show that explicitly enforcing structural constraints allows for the deterministic rejection of this specific class of hallucinations, serving as a neuro-symbolic sanity check for generative reasoning.

Figures (4)

• Figure 1: Overview of Eidoku. System 1 (LLM) generates candidate reasoning sequences. Eidoku (System 2) assigns each candidate an additive Violation Cost composed of independent proxies (structure, local geometry, and logic), applies a rejection rule based on a calibrated critical threshold $\tau_c$, and selects the lowest-score accepted candidate.
• Figure 2: Cumulative semantic Violation Cost across reasoning steps. A candidate is rejected when Violation Cost exceeds the calibrated critical threshold $\tau_c$, enforcing closure by disallowing semantically unconstrained “escape routes.”
• Figure 3: Evaluation pipeline on the Reasoning Gap Dataset (RGD). Given a context forming a valid logical chain ($A \rightarrow B \rightarrow C$), models evaluate a structurally valid conclusion ($A \rightarrow C$) and a semantically plausible but unsupported conclusion ($A \rightarrow D$). Probabilistic baselines rely on semantic similarity, whereas Eidoku evaluates structural connectivity in the induced reasoning graph.
• Figure 4: Sensitivity of Safety (FTAR) and Utility (TTAR) to the calibration percentile $p$ (with fixed $\delta=0.1$). $\delta$ provides a sufficient noise floor; results are insensitive beyond this point. We further performed a 2D grid search over $p \in [85, 99]$ and margin $\delta \in [0.0, 0.3]$. We observed $\delta \approx 0.1$, where the margin effectively covers baseline embedding noise. Crucially, for all $\delta \ge 0.1$, performance remained **invariant** (FTAR=0.00, TTAR=1.00) across the entire range of $p$. This confirms that $\delta=0.1$ is not a narrowly tuned sweet spot, but simply a sufficient noise floor to ensure structural separability.