LIA: Supervised Fine-Tuning of Large Language Models for Automatic Issue Assignment
Arsham Khosravani, Alireza Hosseinpour, Arshia Akhavan, Mehdi Keshani, Abbas Heydarnoori
TL;DR
This paper tackles the challenge of automatic issue assignment in large software projects by fine-tuning an open-source large language model (DeepSeek-R1-Distill-Llama-8B) on historical issue–assignee pairs. By converting issue reports into instructional prompts and training with a causal language modeling objective, LIA learns to generate ranked assignee recommendations from the issue text alone, without relying on explicit graph structures. Empirical results on EclipseJDT and Mozilla show LIA substantially outperforms its base pretrained model and four state-of-the-art baselines, especially for top-ranked predictions (Hit@1), with gains up to +187.8% over the base and up to +211.2% over baselines. The findings demonstrate the practicality and effectiveness of domain-adapted LLMs for software maintenance tasks and suggest promising future directions such as retrieval-augmented generation and broader cross-project evaluation.
Abstract
Issue assignment is a critical process in software maintenance, where new issue reports are validated and assigned to suitable developers. However, manual issue assignment is often inconsistent and error-prone, especially in large open-source projects where thousands of new issues are reported monthly. Existing automated approaches have shown promise, but many rely heavily on large volumes of project-specific training data or relational information that is often sparse and noisy, which limits their effectiveness. To address these challenges, we propose LIA (LLM-based Issue Assignment), which employs supervised fine-tuning to adapt an LLM, DeepSeek-R1-Distill-Llama-8B in this work, for automatic issue assignment. By leveraging the LLM's pretrained semantic understanding of natural language and software-related text, LIA learns to generate ranked developer recommendations directly from issue titles and descriptions. The ranking is based on the model's learned understanding of historical issue-to-developer assignments, using patterns from past tasks to infer which developers are most likely to handle new issues. Through comprehensive evaluation, we show that LIA delivers substantial improvements over both its base pretrained model and state-of-the-art baselines. It achieves up to +187.8% higher Hit@1 compared to the DeepSeek-R1-Distill-Llama-8B pretrained base model, and outperforms four leading issue assignment methods by as much as +211.2% in Hit@1 score. These results highlight the effectiveness of domain-adapted LLMs for software maintenance tasks and establish LIA as a practical, high-performing solution for issue assignment.