AILINKPREVIEWER: Enhancing Code Reviews with LLM-Powered Link Previews

TL;DR

The paper tackles cognitive load in code reviews caused by external links in PRs by introducing AILinkPreviewer, an LLM-powered extension that generates contextual previews from PR metadata. It compares three approaches—Contextual LLM, Non-Contextual LLM, and Metadata Snippets—using a dataset of 50 engineered GitHub repositories and evaluates via BLEU, ROUGE, METEOR, BERTScore, readability, and compression, plus a small user study. Contextual LLM previews achieve higher lexical and semantic quality and link relevance than baselines, though reviewers favored non-contextual previews for usability in a small study. The tool demonstrates potential to reduce context switching and enrich automated review tasks, with future work to refine prompts and navigation within linked pages.

Abstract

Code review is a key practice in software engineering, where developers evaluate code changes to ensure quality and maintainability. Links to issues and external resources are often included in Pull Requests (PRs) to provide additional context, yet they are typically discarded in automated tasks such as PR summarization and code review comment generation. This limits the richness of information available to reviewers and increases cognitive load by forcing context-switching. To address this gap, we present AILINKPREVIEWER, a tool that leverages Large Language Models (LLMs) to generate previews of links in PRs using PR metadata, including titles, descriptions, comments, and link body content. We analyzed 50 engineered GitHub repositories and compared three approaches: Contextual LLM summaries, Non-Contextual LLM summaries, and Metadata-based previews. The results in metrics such as BLEU, BERTScore, and compression ratio show that contextual summaries consistently outperform other methods. However, in a user study with seven participants, most preferred non-contextual summaries, suggesting a trade-off between metric performance and perceived usability. These findings demonstrate the potential of LLM-powered link previews to enhance code review efficiency and to provide richer context for developers and automation in software engineering. The video demo is available at https://www.youtube.com/watch?v=h2qH4RtrB3E, and the tool and its source code can be found at https://github.com/c4rtune/AILinkPreviewer.

Figures (3)

• Figure 1: A PR containing numerous links to external resources
• Figure 2: Architecture for AILinkPreviewer extension
• Figure 3: Example of AILinkPreviewer popup modal