Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Large Language Models for Realizing Truly Intelligent User Interfaces

Allard Oelen, Sören Auer

TL;DR

This work addresses the challenge of organizing rapidly growing scholarly knowledge by leveraging knowledge graphs and semantic representations, and it introduces non-intrusive, LLM-guided UI components to assist users in transforming unstructured article content. The authors present 22 implementable guidelines across six usability pillars, translate these into concrete requirements, and implement them in the Open Research Knowledge Graph (ORKG) UI as Smart Suggestions. An initial small-scale evaluation with domain experts indicates that the guidelines are generally valued, the use cases are useful, and users appreciate AI-assisted guidance while still relying on human validation. The approach emphasizes a conservative, human-in-the-loop paradigm, aiming for broad applicability beyond scholarly domains and setting the stage for larger-scale real-world assessments and exploration of alternative models.

Abstract

The number of published scholarly articles is growing at a significant rate, making scholarly knowledge organization increasingly important. Various approaches have been proposed to organize scholarly information, including describing scholarly knowledge semantically leveraging knowledge graphs. Transforming unstructured knowledge, presented within articles, to structured and semantically represented knowledge generally requires human intelligence and labor since natural language processing methods alone typically do not render sufficient precision and recall for many applications. With the recent developments of Large Language Models (LLMs), it becomes increasingly possible to provide truly intelligent user interfaces guiding humans in the transformation process. We present an approach to integrate non-intrusive LLMs guidance into existing user interfaces. More specifically, we integrate LLM-supported user interface components into an existing scholarly knowledge infrastructure. Additionally, we provide our experiences with LLM integration, detailing best practices and obstacles. Finally, we evaluate the approach using a small-scale user evaluation with domain experts.

Leveraging Large Language Models for Realizing Truly Intelligent User Interfaces

TL;DR

This work addresses the challenge of organizing rapidly growing scholarly knowledge by leveraging knowledge graphs and semantic representations, and it introduces non-intrusive, LLM-guided UI components to assist users in transforming unstructured article content. The authors present 22 implementable guidelines across six usability pillars, translate these into concrete requirements, and implement them in the Open Research Knowledge Graph (ORKG) UI as Smart Suggestions. An initial small-scale evaluation with domain experts indicates that the guidelines are generally valued, the use cases are useful, and users appreciate AI-assisted guidance while still relying on human validation. The approach emphasizes a conservative, human-in-the-loop paradigm, aiming for broad applicability beyond scholarly domains and setting the stage for larger-scale real-world assessments and exploration of alternative models.

Abstract

The number of published scholarly articles is growing at a significant rate, making scholarly knowledge organization increasingly important. Various approaches have been proposed to organize scholarly information, including describing scholarly knowledge semantically leveraging knowledge graphs. Transforming unstructured knowledge, presented within articles, to structured and semantically represented knowledge generally requires human intelligence and labor since natural language processing methods alone typically do not render sufficient precision and recall for many applications. With the recent developments of Large Language Models (LLMs), it becomes increasingly possible to provide truly intelligent user interfaces guiding humans in the transformation process. We present an approach to integrate non-intrusive LLMs guidance into existing user interfaces. More specifically, we integrate LLM-supported user interface components into an existing scholarly knowledge infrastructure. Additionally, we provide our experiences with LLM integration, detailing best practices and obstacles. Finally, we evaluate the approach using a small-scale user evaluation with domain experts.
Paper Structure (15 sections, 4 figures, 2 tables)

This paper contains 15 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Approach and Implementation
  3. Guidelines
  4. Requirements Implementation
  5. Use Cases
  6. Technical Details
  7. Evaluation
  8. Demographics and Setup
  9. Results
  10. Approach Evaluation
  11. Use Case Evaluation
  12. Attitudes Evaluation
  13. Discussion
  14. Conclusion
  15. Use Cases and Prompts

Figures (4)

  • Figure 1: The two main components of the UI implementation.
  • Figure 2: Approach evaluation results. The individual guidelines are evaluated by means of importance according to the participants.
  • Figure 3: Results of the usefulness and correctness aggregated for six different use cases and three different domains.
  • Figure 4: Evaluation results of the participants' attitudes towards the Smart Suggestions as implemented in the ORKG. Additionally, it shows their opinions and experiences using ChatGPT.