The Discovery Engine: A Framework for AI-Driven Synthesis and Navigation of Scientific Knowledge Landscapes

TL;DR

The paper tackles information overload and reproducibility crises by introducing the Discovery Engine (DE) and its core Conceptual Nexus Model (CNM), a computable, tensor-based representation that distills literature into verifiable knowledge artifacts. It proposes guided AI distillation with adaptive templates to produce structured components linked to source evidence, encoded in the Conceptual Nexus Tensor $T_{\text{CNM}}$ and accessible via CNM graphs and semantic vector views. AI agents operate on this structured representation to uncover non-obvious connections, identify knowledge gaps, and generate Knowledge Artifacts such as hypotheses and experimental designs, enabling a shift from document-centric to knowledge-centric science. The framework is validated conceptually through case studies and articulated as a scalable, FAIR-aligned platform with future directions toward integrated experimentation and automated discovery, aiming to transform how scientific knowledge is synthesized, navigated, and created.

Abstract

The prevailing model for disseminating scientific knowledge relies on individual publications dispersed across numerous journals and archives. This legacy system is ill suited to the recent exponential proliferation of publications, contributing to insurmountable information overload, issues surrounding reproducibility and retractions. We introduce the Discovery Engine, a framework to address these challenges by transforming an array of disconnected literature into a unified, computationally tractable representation of a scientific domain. Central to our approach is the LLM-driven distillation of publications into structured "knowledge artifacts," instances of a universal conceptual schema, complete with verifiable links to source evidence. These artifacts are then encoded into a high-dimensional Conceptual Tensor. This tensor serves as the primary, compressed representation of the synthesized field, where its labeled modes index scientific components (concepts, methods, parameters, relations) and its entries quantify their interdependencies. The Discovery Engine allows dynamic "unrolling" of this tensor into human-interpretable views, such as explicit knowledge graphs (the CNM graph) or semantic vector spaces, for targeted exploration. Crucially, AI agents operate directly on the graph using abstract mathematical and learned operations to navigate the knowledge landscape, identify non-obvious connections, pinpoint gaps, and assist researchers in generating novel knowledge artifacts (hypotheses, designs). By converting literature into a structured tensor and enabling agent-based interaction with this compact representation, the Discovery Engine offers a new paradigm for AI-augmented scientific inquiry and accelerated discovery.

Figures (8)

• Figure 1: Conceptual Nexus Model for distillation of the knowledge into machine-readable format ready for human and agent exploration and machine-facilitated discoveries.
• Figure 2: The self-consistent template refinement cycle in the DE until the template and the corps of literature become consistent.
• Figure 3: Conceptual architecture of the Discovery Engine framework.
• Figure 4: User Interface (UI) for the DE platform, fully generated through an AI-assisted design process. The core modules (e.g., graph visualization, knowledge browser, agent interaction panel, hypothesis workbench) and their relationships were initially outlined by DE (with Gemini Pro 2.5) synthesizing best practices from HCI and KG interaction literature, then refined by a human.
• Figure 5: Conceptual architecture of the Distillation Template.