The Role of Visualization in LLM-Assisted Knowledge Graph Systems: Effects on User Trust, Exploration, and Workflows

TL;DR

This paper investigates how visualization influences user trust, exploration strategies, and decision-making in LLM-assisted knowledge-graph exploration. It introduces LinkQ, a system that converts natural-language questions into KG queries via an LLM and augments the workflow with visual mechanisms to expose potential LLM and KG errors. Through a qualitative study with 14 practitioners and a small quantitative evaluation on 120 Mintaka questions, the authors show that while visualizations can increase engagement and trust, they can also induce overtrust when LLM outputs are incorrect. The work provides design guidelines and discusses extensions to mitigate overtrust, improve contextualization, and adapt to user expertise, underscoring the nuanced role of visualization in LLM-KG systems and its practical implications for trustworthy data analysis.

Abstract

Knowledge graphs (KGs) are powerful data structures, but exploring them effectively remains difficult for even expert users. Large language models (LLMs) are increasingly used to address this gap, yet little is known empirically about how their usage with KGs shapes user trust, exploration strategies, or downstream decision-making - raising key design challenges for LLM-based KG visual analysis systems. To study these effects, we developed LinkQ, a KG exploration system that converts natural language questions into structured queries with an LLM. We collaborated with KG experts to design five visual mechanisms that help users assess the accuracy of both KG queries and LLM responses: an LLM-KG state diagram that illustrates which stage of the exploration pipeline LinkQ is in, a query editor displaying the generated query paired with an LLM explanation, an entity-relation ID table showing extracted KG entities and relations with semantic descriptions, a query structure graph that depicts the path traversed in the KG, and an interactive graph visualization of query results. From a qualitative evaluation with 14 practitioners, we found that users - even KG experts - tended to overtrust LinkQ's outputs due to its "helpful" visualizations, even when the LLM was incorrect. Users exhibited distinct workflows depending on their prior familiarity with KGs and LLMs, challenging the assumption that these systems are one-size-fits-all - despite often being designed as if they are. Our findings highlight the risks of false trust in LLM-assisted data analysis tools and the need for further investigation into the role of visualization as a mitigation technique.

Figures (4)

• Figure 1: A high-level overview of LinkQ'sli2024linkq prompting protocol, described in Section \ref{['sec:prompting-protocol']}. For this pipeline, the LLM, System, and User have precise responsibilities for completing the question-to-query translation.
• Figure 2: An example of the LinkQ prompting protocol described in Section \ref{['sec:prompting-protocol']}, based off the scenario in Figure \ref{['fig:teaser']}. Initially, the LLM iterates with the User to clarify their question. Then the LLM searches for ground-truth data in the KG. Once the LLM has found IDs it needs, it generates a query, then summarizes and LinkQ visualizes the results. Each stage of the workflow is depicted in a state diagram at the top of LinkQ, and users can choose to view this full conversation history in the user interface. On the right side of the figure, we note possible LLM failure points in the process.
• Figure 3: Example of LinkQ's Results Graph Visualization, where nodes (represented as a table of entities) and edges (the relations connecting entities) are extracted from the KG.
• Figure 4: Process of our qualitative user study (Section \ref{['sec:qualitative-user-evaluation']}). Participants completed a series of tasks with LinkQ using two different KGs, beginning with Wikidata (a general-purpose KG) and ending with BRON (a domain-specific cybersecurity KG).