Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Speeding up Program Repair with Non-Autoregressive Model

Zhenyu Yang, Yue Pan, Zhen Yang, Zhongxing Yu

TL;DR

This paper tackles the speed bottleneck of machine learning-based automatic program repair by introducing NARRepair, a non-autoregressive code-generation model. It tackles three key AR pitfalls with a repair action predictor, an inter-token dependency extractor leveraging AST structure, and a two-stage decoder to restore contextual information, enabling parallel patch generation. Evaluations on Defects4J v1.2, Defects4J v2.0, and QuixBugs show that NARRepair achieves state-of-the-art speed and accuracy, delivering $1.4$ to $6.4$× faster repair than AR baselines while fixing more bugs within tight time limits, using a compact 520M parameter model. These results demonstrate the practical viability of speed-optimized NP architectures for real-time APR and motivate applying similar speed-oriented design choices to broader software engineering tasks.

Abstract

Enlightened by the success of machine learning techniques in various application areas, recent years have witnessed a surge of research efforts on automatic program repair (APR) using machine learning techniques. Previous machine learning-based APR techniques essentially modified bugs in the autoregressive (AR) manner, which predicts future values based on past values. Due to the manner of token-by-token generation, the AR-based APR technique has a huge time delay. In particular, the delay of the APR model with a large number of parameters is more serious. To address the issue, we aim to apply the non-autoregressive (NAR) method to the APR task, which can output target code in a parallel manner to avoid huge repair delays. However, the naive use of the NAR manner for the APR task suffers from the issue of compromised patch quality. To effectively adapt the NAR manner for the APR task, we in this paper propose NARRepair, the first customized NAR code generation model for the APR task. The NARRepair model features three major novelties, including 1) the repair action predictor for alleviating the over-correction issue, 2) the inter-token dependency extractor for alleviating the issue of lacking inter-token dependency information, and 3) the two-stage decoder for alleviating the issue of lacking contextual information. We evaluated NARRepair on three widely used datasets in the APR community, and the results show that 1) compared to other APR techniques, the NARRepair model has the best performance within the limited repair time, and 2) compared to AR-based APR techniques, the repair speed of NARRepair has been increased by 1.4-6.4 times in the GPU environment. Overall, the results show that NARRepair has achieved state-of-the-art comprehensive performance in terms of repair speed and accuracy.

Towards Speeding up Program Repair with Non-Autoregressive Model

TL;DR

This paper tackles the speed bottleneck of machine learning-based automatic program repair by introducing NARRepair, a non-autoregressive code-generation model. It tackles three key AR pitfalls with a repair action predictor, an inter-token dependency extractor leveraging AST structure, and a two-stage decoder to restore contextual information, enabling parallel patch generation. Evaluations on Defects4J v1.2, Defects4J v2.0, and QuixBugs show that NARRepair achieves state-of-the-art speed and accuracy, delivering to × faster repair than AR baselines while fixing more bugs within tight time limits, using a compact 520M parameter model. These results demonstrate the practical viability of speed-optimized NP architectures for real-time APR and motivate applying similar speed-oriented design choices to broader software engineering tasks.

Abstract

Enlightened by the success of machine learning techniques in various application areas, recent years have witnessed a surge of research efforts on automatic program repair (APR) using machine learning techniques. Previous machine learning-based APR techniques essentially modified bugs in the autoregressive (AR) manner, which predicts future values based on past values. Due to the manner of token-by-token generation, the AR-based APR technique has a huge time delay. In particular, the delay of the APR model with a large number of parameters is more serious. To address the issue, we aim to apply the non-autoregressive (NAR) method to the APR task, which can output target code in a parallel manner to avoid huge repair delays. However, the naive use of the NAR manner for the APR task suffers from the issue of compromised patch quality. To effectively adapt the NAR manner for the APR task, we in this paper propose NARRepair, the first customized NAR code generation model for the APR task. The NARRepair model features three major novelties, including 1) the repair action predictor for alleviating the over-correction issue, 2) the inter-token dependency extractor for alleviating the issue of lacking inter-token dependency information, and 3) the two-stage decoder for alleviating the issue of lacking contextual information. We evaluated NARRepair on three widely used datasets in the APR community, and the results show that 1) compared to other APR techniques, the NARRepair model has the best performance within the limited repair time, and 2) compared to AR-based APR techniques, the repair speed of NARRepair has been increased by 1.4-6.4 times in the GPU environment. Overall, the results show that NARRepair has achieved state-of-the-art comprehensive performance in terms of repair speed and accuracy.

Paper Structure

This paper contains 26 sections, 12 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automatic Program Repair
  4. Machine Learning-based Repair
  5. Non-autoregressive Models
  6. NARRepair
  7. Code Encoder
  8. Repair Action Predictor
  9. Inter-token Dependency Extractor
  10. Two-stage NAR Decoder
  11. Training and Inference
  12. Evaluation Setup
  13. Research Questions
  14. Data Collection
  15. Data Preprocessing
  16. ...and 11 more sections

Figures (8)

  • Figure 1: The Inference Processes of the Autoregressive (AR) Model and the Non-autoregressive (NAR) Model.
  • Figure 2: An Overview of the NARRepair Architecture.
  • Figure 3: An Example of Repair Action Prediction for a Real Bug.
  • Figure 4: The Example of Generating AST and Inter-token Dependency Matrix for the Code return a+b;.
  • Figure 5: The Structure of the Inter-token Dependency Extractor.
  • ...and 3 more figures