Introducing ORKG ASK: an AI-driven Scholarly Literature Search and Exploration System Taking a Neuro-Symbolic Approach

TL;DR

The paper presents ASK, an open-source, neuro-symbolic scholarly search system that combines vector-based retrieval, LLM-based information extraction, and knowledge-graph semantics via a Retrieval-Augmented Generation framework. It details the architecture, functional and non-functional requirements, implementation choices, and an evaluation that includes both subjective usability metrics and objective production analytics. The study shows ASK can reduce user task load compared to traditional search while maintaining transparency about AI provenance and encouraging human verification. The work suggests practical impact in enabling reproducible, accessible, and scalable literature exploration, with future work aimed at expanding corpus diversity and semantic capabilities.

Abstract

As the volume of published scholarly literature continues to grow, finding relevant literature becomes increasingly difficult. With the rise of generative Artificial Intelligence (AI), and particularly Large Language Models (LLMs), new possibilities emerge to find and explore literature. We introduce ASK (Assistant for Scientific Knowledge), an AI-driven scholarly literature search and exploration system that follows a neuro-symbolic approach. ASK aims to provide active support to researchers in finding relevant scholarly literature by leveraging vector search, LLMs, and knowledge graphs. The system allows users to input research questions in natural language and retrieve relevant articles. ASK automatically extracts key information and generates answers to research questions using a Retrieval-Augmented Generation (RAG) approach. We present an evaluation of ASK, assessing the system's usability and usefulness. Findings indicate that the system is user-friendly and users are generally satisfied while using the system.

Figures (6)

• Figure 1: Explainer depicting our RAG (Retrieval-Augmented Generation) approach for scholarly search. The Retrieval step ranks articles by their relevance to the question. The Augmented step injects the previously retrieved context in the prompt. The Generation step prompts the LLM and displays the answer.
• Figure 2: Screenshot of the ASK search results page. The nodes with (N)FR correspond to the implementation of the (Non-)functional requirements, as listed in \ref{['table:requirements']}.
• Figure 3: Sample prompts for different RAG use cases within ASK. The system prompts provide the instructions to the LLM (Prompt P1 trimmed for brevity reasons). The user prompt includes the user input and RAG context. Values highlighted in red are placeholders used to inject user values into the prompt. Additionally, P2 uses a primer to improve the answer of the LLM.
• Figure 4: Question specific results for operational feedback collection.
• Figure 5: UMUX-Lite results with a score of 65.7.