Toward Linking Declined Proposals and Source Code: An Exploratory Study on the Go Repository

TL;DR

The paper tackles the challenge of making the knowledge embedded in declined software proposals accessible by linking them to source code. It introduces an LLM-driven, granularity-aware pipeline that first decides the code granularity and then localizes and links relevant code artifacts, evaluated on Go language proposals. On declined proposals, the pipeline achieves an overall granularity accuracy of 0.836 and a mean link precision of 0.643, with stronger performance at file than function granularity. A failure analysis reveals that missing concrete implementation details and obscured signals in discussions largely drive failures, guiding future augmentation and summarization approaches to improve traceability in unimplemented design discussions.

Abstract

Traceability links are key information sources for software developers, connecting software artifacts (e.g., linking requirements to the corresponding source code). In open-source software (OSS) projects, such links play an important role, particularly between the contributions (e.g., GitHub issues) and the corresponding source code. Through these links, developers can trace the discussions in contributions and uncover design rationales, constraints, and security concerns. Previous studies have mainly examined accepted contributions, while those declined after discussion have been overlooked. The discussions behind declined contributions contain valuable design rationales and implicit knowledge about software decision-making, as the reasons behind the decline often reveal the criteria used to judge what should or should not be implemented. In this study, we present the first attempt to establish traceability links between declined contributions and related source code. We propose an initial linking approach and conduct an empirical analysis of the generated links to discuss factors affecting link generation. As our dataset, we use proposals from the official Go repository, which are GitHub issues used to propose new features or language changes. To link declined proposals to source code, we designed an LLM-driven pipeline. Our results showed that the pipeline selected the correct granularity for each declined proposal with an accuracy of 0.836, and generated correct links at that granularity with a mean precision of 0.643. To clarify the challenges of linking declined proposals, we performed a failure analysis. In the declined proposals where the pipeline failed to generate links, the discussions were often redundant and lacked concrete information (e.g., how the feature should be implemented).

Figures (9)

• Figure 1: Development workflow of proposals
• Figure 2: LLM-driven pipeline for generating granularity-aware traceability links
• Figure 3: Input–output example retrieved from issues/14757 in golang/go issue_14757
• Figure 4: Three levels of granularity for declined proposals
• Figure 5: Comparison between LLM-based and embedding-based localization across three metrics