Supporting Software Maintenance with Dynamically Generated Document Hierarchies

TL;DR

This paper tackles the heavy burden of producing multi-level software documentation for maintenance tasks. It introduces HGEN, a six-stage, LLM-assisted pipeline that transforms source code into a hierarchical, well-structured artifact tree with trace links across layers. Through controlled comparisons on three projects and nine industrial pilots, HGEN demonstrates documentation quality on par with human-created material and superior coverage of core concepts, while delivering strong traceability between artifacts. The findings suggest substantial practical impact for faster code comprehension, onboarding, and maintenance, with concrete pathways for integrating HGEN into real-world workflows and future automation of requirements engineering.

Abstract

Software documentation supports a broad set of software maintenance tasks; however, creating and maintaining high-quality, multi-level software documentation can be incredibly time-consuming and therefore many code bases suffer from a lack of adequate documentation. We address this problem through presenting HGEN, a fully automated pipeline that leverages LLMs to transform source code through a series of six stages into a well-organized hierarchy of formatted documents. We evaluate HGEN both quantitatively and qualitatively. First, we use it to generate documentation for three diverse projects, and engage key developers in comparing the quality of the generated documentation against their own previously produced manually-crafted documentation. We then pilot HGEN in nine different industrial projects using diverse datasets provided by each project. We collect feedback from project stakeholders, and analyze it using an inductive approach to identify recurring themes. Results show that HGEN produces artifact hierarchies similar in quality to manually constructed documentation, with much higher coverage of the core concepts than the baseline approach. Stakeholder feedback highlights HGEN's commercial impact potential as a tool for accelerating code comprehension and maintenance tasks. Results and associated supplemental materials can be found at https://zenodo.org/records/11403244

Figures (7)

• Figure 1: The HGEN Process utilizes a pipeline to produce each layer of the documentation hierarchy. The lowest layer accepts source code as input and generates a natural language summary (Step 0). Steps 1-5 form a pipeline, which is used to generate each subsequent layer, thereby incrementally constructing a hierarchy of progressively higher-level artifacts formatted according to the norms of the current software development process. For each step in the pipeline, we show the underlying AI models used to support the transformation of lower-level to higher-level documentation.
• Figure 2: As part of our running example, HGEN summarizes the HERO source code in the preprocessing Stage 0.
• Figure 3: In Stage 1, HGEN uses multiple clustering algorithms to produce and initial set of clusters from the source code summaries, and then performs a series of filtering, ranking, and cleansing steps on the clusters.
• Figure 4: User stories generated from HERO source code during Stage 2. All 3 user stories were produced from the same cluster of source artifacts (see Cluster 5 from Figure \ref{['fig:cluster_example']}).
• Figure 6: Refined User Stories for HERO during Stage 3