Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Drawing for LLMs: An Empirical Evaluation

Walter Didimo, Fabrizio Montecchiani, Tommaso Piselli

TL;DR

This paper investigates how Large Language Models solve graph-related tasks when given graph drawings as input, examining the influence of layout paradigms (orthogonal vs straight-line), prompting strategies (Std, CoT, and SoAL), and input readability on performance. It conducts three experiments using GPT-4o and Claude-3.7-Sonnet across textual, visual, and mixed modalities, including a novel SoAL prompting method and human-readability-based layout improvements. The results show orthogonal layouts excel for local connectivity tasks, while straight-line drawings better reveal global structure; prompting diversity helps, with CoT generally effective and SoAL offering promise in certain models. The findings provide practical guidance for designing AI-assisted graph tools and suggest avenues for incorporating readability metrics into graph-aware multimodal reasoning.

Abstract

Our work contributes to the fast-growing literature on the use of Large Language Models (LLMs) to perform graph-related tasks. In particular, we focus on usage scenarios that rely on the visual modality, feeding the model with a drawing of the graph under analysis. We investigate how the model's performance is affected by the chosen layout paradigm, the aesthetics of the drawing, and the prompting technique used for the queries. We formulate three corresponding research questions and present the results of a thorough experimental analysis. Our findings reveal that choosing the right layout paradigm and optimizing the readability of the input drawing from a human perspective can significantly improve the performance of the model on the given task. Moreover, selecting the most effective prompting technique is a challenging yet crucial task for achieving optimal performance.

Graph Drawing for LLMs: An Empirical Evaluation

TL;DR

This paper investigates how Large Language Models solve graph-related tasks when given graph drawings as input, examining the influence of layout paradigms (orthogonal vs straight-line), prompting strategies (Std, CoT, and SoAL), and input readability on performance. It conducts three experiments using GPT-4o and Claude-3.7-Sonnet across textual, visual, and mixed modalities, including a novel SoAL prompting method and human-readability-based layout improvements. The results show orthogonal layouts excel for local connectivity tasks, while straight-line drawings better reveal global structure; prompting diversity helps, with CoT generally effective and SoAL offering promise in certain models. The findings provide practical guidance for designing AI-assisted graph tools and suggest avenues for incorporating readability metrics into graph-aware multimodal reasoning.

Abstract

Our work contributes to the fast-growing literature on the use of Large Language Models (LLMs) to perform graph-related tasks. In particular, we focus on usage scenarios that rely on the visual modality, feeding the model with a drawing of the graph under analysis. We investigate how the model's performance is affected by the chosen layout paradigm, the aesthetics of the drawing, and the prompting technique used for the queries. We formulate three corresponding research questions and present the results of a thorough experimental analysis. Our findings reveal that choosing the right layout paradigm and optimizing the readability of the input drawing from a human perspective can significantly improve the performance of the model on the given task. Moreover, selecting the most effective prompting technique is a challenging yet crucial task for achieving optimal performance.
Paper Structure (17 sections, 12 figures, 9 tables)

This paper contains 17 sections, 12 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Black-box models
  4. Ad-hoc models
  5. Experimental analysis
  6. Experiment 1: Comparing multiple drawing paradigms
  7. Experimental set-up
  8. Results
  9. Experiment 2: Introducing a new prompting technique
  10. Experimental set-up
  11. Results
  12. Experiment 3: Evaluating the impact of improved quality metrics
  13. Experimental set-up
  14. Results
  15. Conclusions
  16. ...and 2 more sections

Figures (12)

  • Figure 1: High-level architecture of our experimental framework.
  • Figure 2: Examples of drawings computed for the SlV (left) and OrV (right) modalities. The first row shows a graph from Bench-1, the second row shows a graph from Bench-2 (with a max clique of size five), the third row shows a graph from Bench-3 (with a min independent set of size six).
  • Figure 3: Experiment 1: Average accuracy by modality.
  • Figure 4: Experiment 1: Average accuracy by modality per task.
  • Figure 5: Experiment 1: Average number of total tokens by modality.
  • ...and 7 more figures