Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts

Beian Wang, Chong Wang, Peng Liang, Bing Li, Cheng Zeng

TL;DR

The paper investigates the feasibility of using large language models to assist novice analysts in requirements engineering by generating three common UML diagrams (use case, class, and sequence). An empirical study with 45 undergraduates evaluates LLM-assisted modeling and the effect of output formats, revealing that LLMs can help with identifying modeling elements but have notable weaknesses in capturing relationships, with sequence diagrams showing relatively higher accuracy. Hybrid human-LLM workflows yield the best quality, suggesting that human review remains important for reliable UML modeling. The work provides actionable guidance for educators and practitioners on when and how to deploy LLM-based assistance in requirements analysis and UML modeling, and it outlines concrete directions to improve reliability and format choice in future research.

Abstract

Since the emergence of GPT-3, Large Language Models (LLMs) have caught the eyes of researchers, practitioners, and educators in the field of software engineering. However, there has been relatively little investigation regarding the performance of LLMs in assisting with requirements analysis and UML modeling. This paper explores how LLMs can assist novice analysts in creating three types of typical UML models: use case models, class diagrams, and sequence diagrams. For this purpose, we designed the modeling tasks of these three UML models for 45 undergraduate students who participated in a requirements modeling course, with the help of LLMs. By analyzing their project reports, we found that LLMs can assist undergraduate students as novice analysts in UML modeling tasks, but LLMs also have shortcomings and limitations that should be considered when using them.

How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts

TL;DR

The paper investigates the feasibility of using large language models to assist novice analysts in requirements engineering by generating three common UML diagrams (use case, class, and sequence). An empirical study with 45 undergraduates evaluates LLM-assisted modeling and the effect of output formats, revealing that LLMs can help with identifying modeling elements but have notable weaknesses in capturing relationships, with sequence diagrams showing relatively higher accuracy. Hybrid human-LLM workflows yield the best quality, suggesting that human review remains important for reliable UML modeling. The work provides actionable guidance for educators and practitioners on when and how to deploy LLM-based assistance in requirements analysis and UML modeling, and it outlines concrete directions to improve reliability and format choice in future research.

Abstract

Since the emergence of GPT-3, Large Language Models (LLMs) have caught the eyes of researchers, practitioners, and educators in the field of software engineering. However, there has been relatively little investigation regarding the performance of LLMs in assisting with requirements analysis and UML modeling. This paper explores how LLMs can assist novice analysts in creating three types of typical UML models: use case models, class diagrams, and sequence diagrams. For this purpose, we designed the modeling tasks of these three UML models for 45 undergraduate students who participated in a requirements modeling course, with the help of LLMs. By analyzing their project reports, we found that LLMs can assist undergraduate students as novice analysts in UML modeling tasks, but LLMs also have shortcomings and limitations that should be considered when using them.
Paper Structure (19 sections, 3 equations, 5 figures, 8 tables)

This paper contains 19 sections, 3 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Research Process
  4. Research Objective
  5. Research Questions
  6. Experiment Design
  7. Phase 1: Setting up modeling tasks
  8. Phase 2: Evaluate modeling results
  9. Results
  10. Demographics of the Participants
  11. Answer to RQ1: Use Case Diagram Modeling
  12. Answer to RQ2: Class Diagram Modeling
  13. Answer to RQ3: Sequence Diagram Modeling
  14. Answer to RQ4: The Impact of Output Format on the Modeling Performance of LLMs
  15. Discussion
  16. ...and 4 more sections

Figures (5)

  • Figure 1: The process of the experiment
  • Figure 2: Examples of the three output formats of LLMs
  • Figure 3: Distribution of the participants with/without experience of using LLMs
  • Figure 4: Distribution of the LLMs used in UML modeling tasks
  • Figure 5: Distribution of the languages used in the human-LLM interaction