Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge Acquisition and Integration with Expert-in-the-loop

Sajjadur Rahman, Frederick Choi, Hannah Kim, Dan Zhang, Estevam Hruschka

TL;DR

The paper addresses the challenge of continuous, human-in-the-loop knowledge graph construction in enterprise settings. It introduces Kyurem, a programmable, interactive widget library embedded in computational notebooks (built on the Magneton framework) that unifies graph visualization, data operations, and AI-assisted recommendations for knowledge acquisition and integration. Through participatory design and two real-world case studies in Company-X's HRKG platform, Kyurem is shown to improve user experience, reduce tedious context switching, and help surface data-quality issues during curation tasks. The work contributes a concrete design process, system architecture, and evaluative evidence that supports broader adoption of in-notebook KG tooling and potential generalization to other data-centric domains.

Abstract

Constructing and serving knowledge graphs (KGs) is an iterative and human-centered process involving on-demand programming and analysis. In this paper, we present Kyurem, a programmable and interactive widget library that facilitates human-in-the-loop knowledge acquisition and integration to enable continuous curation a knowledge graph (KG). Kyurem provides a seamless environment within computational notebooks where data scientists explore a KG to identify opportunities for acquiring new knowledge and verify recommendations provided by AI agents for integrating the acquired knowledge in the KG. We refined Kyurem through participatory design and conducted case studies in a real-world setting for evaluation. The case-studies show that introduction of Kyurem within an existing HR knowledge graph construction and serving platform improved the user experience of the experts and helped eradicate inefficiencies related to knowledge acquisition and integration tasks

Knowledge Acquisition and Integration with Expert-in-the-loop

TL;DR

The paper addresses the challenge of continuous, human-in-the-loop knowledge graph construction in enterprise settings. It introduces Kyurem, a programmable, interactive widget library embedded in computational notebooks (built on the Magneton framework) that unifies graph visualization, data operations, and AI-assisted recommendations for knowledge acquisition and integration. Through participatory design and two real-world case studies in Company-X's HRKG platform, Kyurem is shown to improve user experience, reduce tedious context switching, and help surface data-quality issues during curation tasks. The work contributes a concrete design process, system architecture, and evaluative evidence that supports broader adoption of in-notebook KG tooling and potential generalization to other data-centric domains.

Abstract

Constructing and serving knowledge graphs (KGs) is an iterative and human-centered process involving on-demand programming and analysis. In this paper, we present Kyurem, a programmable and interactive widget library that facilitates human-in-the-loop knowledge acquisition and integration to enable continuous curation a knowledge graph (KG). Kyurem provides a seamless environment within computational notebooks where data scientists explore a KG to identify opportunities for acquiring new knowledge and verify recommendations provided by AI agents for integrating the acquired knowledge in the KG. We refined Kyurem through participatory design and conducted case studies in a real-world setting for evaluation. The case-studies show that introduction of Kyurem within an existing HR knowledge graph construction and serving platform improved the user experience of the experts and helped eradicate inefficiencies related to knowledge acquisition and integration tasks
Paper Structure (15 sections, 5 figures)

This paper contains 15 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Design Goal Formulation
  4. Contextual Inquiry of Graph Exploration Workflows
  5. Key Takeaways
  6. Design Goals
  7. Kyurem Design
  8. Participatory Design
  9. Kyurem System Architecture
  10. Case Study
  11. Study 1: New Knowledge Acquisition
  12. Study 2: Knowledge Integration
  13. Discussion
  14. Conclusion
  15. Kyurem Components Design via Participatory Sessions

Figures (5)

  • Figure 1: (a) An overview of the Company-X HRKG tasks, (b) the study design, and (c) the Kyurem system architecture.
  • Figure 2: Screenshot of a knowledge acquisition workflow using Kyurem widget library. (a, b) importing libraries to launch a multiple-coordinated view widget, (c) leveraging the history view of Magneton to context switch between exploration history, (d, e, f) viewing and exploring multi-modal data interactively and programmaticially.
  • Figure 3: Knowledge acquisition widget: ESE-explorer. Users can launch the interactive widget from a Jupyter notebook, select seed examples of a concept, and export the seeds programmatically.
  • Figure 4: Kyurem base components: (a) faceted graph, (b) node-link diagram, (c) distributions with minimap, (d) distributions, (e) relation degree distributions, (f) hierarchical data viewer, (g) static table, and (h) interactive table.
  • Figure 5: Verification widget for reviewing knowledge integration candidates. Experts may choose a suitable option that reflect their reviewing decision.