Ontology-to-tools compilation for executable semantic constraint enforcement in LLM agents

TL;DR

The paper presents a framework to enforce semantic constraints during LLM-driven knowledge extraction by compiling ontologies into executable tool interfaces within The World Avatar. It demonstrates end-to-end knowledge-graph construction from metal-organic polyhedra synthesis literature, producing grounded synthesis procedures, canonical species, and CBUs, while enabling lexical grounding to reference data sources. End-to-end evaluation on 30 papers shows strong semantic validity and content accuracy, with constraint feedback improving the completeness of synthesis steps; analysis identifies dominant error modes and prioritizes targeted improvements. Limitations include analysis limited to a single ontology and dataset, with plans for broader domain testing, ontology evolution studies, and larger, more diverse evaluation. The approach reframes constraint enforcement as run-time interaction with a persistent symbolic state, enabling ontology-aware, stateful agent behavior in complex scientific extraction tasks.

Abstract

We introduce ontology-to-tools compilation as a proof-of-principle mechanism for coupling large language models (LLMs) with formal domain knowledge. Within The World Avatar (TWA), ontological specifications are compiled into executable tool interfaces that LLM-based agents must use to create and modify knowledge graph instances, enforcing semantic constraints during generation rather than through post-hoc validation. Extending TWA's semantic agent composition framework, the Model Context Protocol (MCP) and associated agents are integral components of the knowledge graph ecosystem, enabling structured interaction between generative models, symbolic constraints, and external resources. An agent-based workflow translates ontologies into ontology-aware tools and iteratively applies them to extract, validate, and repair structured knowledge from unstructured scientific text. Using metal-organic polyhedra synthesis literature as an illustrative case, we show how executable ontological semantics can guide LLM behaviour and reduce manual schema and prompt engineering, establishing a general paradigm for embedding formal knowledge into generative systems.

Figures (10)

• Figure 1: Example ontology instance produced by the instantiation agent. (A) A-Box subgraph instantiated under OntoSyn/OntoMOPs for synthesis S1, including ordered synthesis steps, chemical inputs, and product UMC-1 (with yield and representation links). (B) Compact record projection (A-Box normal form) of the same instance in canonical slot--value form; onsyn:* abbreviates ontosyn:*, and selected ChemicalInput entities are grounded via owl:sameAs to ontospecies:Species.
• Figure 2: Ontology-to-tools compilation as an executable semantic control layer for LLM-based agents. Symbolic ontological definitions (T-Box) within The World Avatar are compiled into executable tool interfaces and validators that define the action space available to a large language model during generation. Rather than producing free-form text, the LLM interacts with a persistent symbolic state by invoking ontology-aligned actions that create, modify, and validate graph instances. Constraint violations trigger structured feedback, enabling iterative repair and grounding to external resources. This reframes semantic constraint enforcement from post-hoc validation or constrained decoding into run-time interaction with an evolving symbolic environment, allowing the model to operate as a stateful, ontology-aware agent.
• Figure 3: Core end-to-end results across four extraction/instantiation domains. (a) Aggregate graph-recoverable precision, recall and F1 for grounded/derived CBUs, characterisation entities, synthesis steps and reaction chemicals (Table \ref{['tab:overall']}). (b) Task imbalance in ground-truth positives (Table \ref{['tab:benchmark_profile']}). (c) Per-paper F1 distributions across the 30-paper benchmark. (d) Per-paper recall variability, highlighting recall-limited categories. (e) Best--worst paper contrast (mean F1 over top-3 vs bottom-3 papers) summarising dataset heterogeneity.
• Figure 4: Component necessity and the role of constraint feedback. (a) Ablation impact on end-to-end F1 by category (Table \ref{['tab:ablation']}). (b) Steps-only precision--recall--F1 shift under feedback removal, illustrating the dominant effect on step completeness/recoverability. (c) Constraint feedback: illustrative failure modes and paired $\Delta$ error counts computed over aligned full vs no-feedback syntheses. The excerpt shows typical no-feedback degradations (e.g. step-number integrity and redundancy), while the mini-plot quantifies three paired error proxies (A--C; defined in-panel) with bootstrap confidence intervals.
• Figure 5: Error anatomy and improvement priorities for synthesis steps. (a) Top error-contributing fields (FP vs FN) aggregated over the benchmark. (b) Field-level bias signature (FN-share vs FP-share), separating recall-limited from precision-limited fields. (c) Hypothetical improvement roadmap (Pareto): cumulative F1 if the top-$N$ error-contributing fields were corrected. (d) Error concentration across papers (Lorenz-style curve), showing whether a small subset of papers accounts for a disproportionate share of step errors.