Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models

Yihan Liu, Zhen Wen, Luoxuan Weng, Ollie Woodman, Yi Yang, Wei Chen

TL;DR

A novel approach that breaks down the programming tutorial creation task into actionable steps is introduced and an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process is presented.

Abstract

The rapid development of large language models (LLMs), such as ChatGPT, has revolutionized the efficiency of creating programming tutorials. LLMs can be instructed with text prompts to generate comprehensive text descriptions of code snippets. However, the lack of transparency in the end-to-end generation process has hindered the understanding of model behavior and limited user control over the generated results. To tackle this challenge, we introduce a novel approach that breaks down the programming tutorial creation task into actionable steps. By employing the tree-of-thought method, LLMs engage in an exploratory process to generate diverse and faithful programming tutorials. We then present SPROUT, an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process. A formal user study demonstrated the effectiveness of SPROUT, showing that our tool assists users to actively participate in the programming tutorial creation process, leading to more reliable and customizable results. By providing users with greater control and understanding, SPROUT enhances the user experience and improves the overall quality of programming tutorial. A free copy of this paper and all supplemental materials are available at https://osf.io/uez2t/?view_only=5102e958802341daa414707646428f86.

SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models

TL;DR

A novel approach that breaks down the programming tutorial creation task into actionable steps is introduced and an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process is presented.

Abstract

The rapid development of large language models (LLMs), such as ChatGPT, has revolutionized the efficiency of creating programming tutorials. LLMs can be instructed with text prompts to generate comprehensive text descriptions of code snippets. However, the lack of transparency in the end-to-end generation process has hindered the understanding of model behavior and limited user control over the generated results. To tackle this challenge, we introduce a novel approach that breaks down the programming tutorial creation task into actionable steps. By employing the tree-of-thought method, LLMs engage in an exploratory process to generate diverse and faithful programming tutorials. We then present SPROUT, an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process. A formal user study demonstrated the effectiveness of SPROUT, showing that our tool assists users to actively participate in the programming tutorial creation process, leading to more reliable and customizable results. By providing users with greater control and understanding, SPROUT enhances the user experience and improves the overall quality of programming tutorial. A free copy of this paper and all supplemental materials are available at https://osf.io/uez2t/?view_only=5102e958802341daa414707646428f86.
Paper Structure (40 sections, 8 figures)

This paper contains 40 sections, 8 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Programming Tutorial Authoring Tools
  4. LLMs for Document Generation
  5. Visual Interfaces for LLMs
  6. Formative Study
  7. Participants and Procedure
  8. Findings
  9. Design Requirements
  10. Prompting for Programming Tutorial Authoring
  11. Generation of Thoughts
  12. Manipulation of Model Actions
  13. Extraction of Text-Code Connections
  14. SPROUT
  15. Overview
  16. ...and 25 more sections

Figures (8)

  • Figure 1: The framework of SPROUT's prompt strategies. We break down the programming tutorial creation task into actionable steps (A) which are provided as initial system prompts to the agent. During the generation process, the agent receives the source code (B) and user instructions, generating multiple potential thoughts (C) based on its memory and the observation derived from the source code and current tutorial content (E). Then it takes the proper action (D) and plans for subsequent steps. The iterative process continues until the tutorial is complete. (F) shows an example iteration.
  • Figure 2: An example output with text-code connection. The agent is instructed to provide text-code connection information within its response.
  • Figure 3: The system interface of SPROUT. The Code View (A) displays the source code. The Tutorial View (B) presents the LLM-generated tutorial content. Between them is the Chain View (C), showing the node chain of current paragraphs selected by users. The Outline (D), Branches (E), and Node Space (F) Views provide interactive visualizations for multi-level modifications such as elaborating, adjusting structure, and polishing content.
  • Figure 4: Select two nodes (A) and group them into one (B).
  • Figure 5: Users can achieve quick assembling nodes by selecting any node in the tree graph as the tail of the chain. For example, after user clicks on node (A1) in outline view, the content in tutorial view (A2) updates correspondingly. (B1) and (B2) is another example.
  • ...and 3 more figures