Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Neural Code Representation with Additional Context

Huy Nguyen, Christoph Treude, Patanamon Thongtanunam

TL;DR

This work addresses the limitation of code-only representations in neural program comprehension by introducing and evaluating context-enhanced encodings that incorporate version history, call graphs, and method age. It conducts an extensive empirical study across two large Java datasets (SeSaMe and CodeSearchNet), five representative models, and three tasks (code clone detection, code classification, and code summarisation), comparing code-only inputs to context-augmented representations. Key findings show that contextual signals generally improve performance, with version history providing the most robust gains and combined contexts yielding the largest improvements in some settings; human evaluation further confirms higher perceived accuracy and adequacy of context-augmented summaries. The results offer practical guidance on when and how to use contextual signals, particularly highlighting the importance of representation-level aggregation and task-specific context selection for real-world software engineering tools and workflows.

Abstract

Automated program comprehension underpins many software engineering tasks, from code summarisation to clone detection. Recent deep learning models achieve strong results but typically rely on source code alone, overlooking contextual information such as version history or structural relationships. This limits their ability to capture how code evolves and operates. We conduct an empirical study on how enriching code representations with such contextual signals affects neural model performance on key comprehension tasks. Two downstream tasks, code clone detection and code summarisation, are evaluated using SeSaMe (1,679 Java methods) and CodeSearchNet (63,259 methods). Five representative models (CodeBERT, GraphCodeBERT, CodeT5, PLBART, ASTNN) are fine-tuned under code-only and context-augmented settings. Results show that context generally improves performance: version history consistently boosts clone detection (e.g., CodeT5 +15.92% F1) and summarisation (e.g., GraphCodeBERT +5.56% METEOR), while call-graph effects vary by model and task. Combining multiple contexts yields further gains (up to +21.48% macro-F1). Human evaluation on 100 Java snippets confirms that context-augmented summaries are significantly preferred for Accuracy and Content Adequacy (p <= 0.026; |delta| up to 0.55). These findings highlight the potential of contextual signals to enhance code comprehension and open new directions for optimising contextual encoding in neural SE models.

Enhancing Neural Code Representation with Additional Context

TL;DR

This work addresses the limitation of code-only representations in neural program comprehension by introducing and evaluating context-enhanced encodings that incorporate version history, call graphs, and method age. It conducts an extensive empirical study across two large Java datasets (SeSaMe and CodeSearchNet), five representative models, and three tasks (code clone detection, code classification, and code summarisation), comparing code-only inputs to context-augmented representations. Key findings show that contextual signals generally improve performance, with version history providing the most robust gains and combined contexts yielding the largest improvements in some settings; human evaluation further confirms higher perceived accuracy and adequacy of context-augmented summaries. The results offer practical guidance on when and how to use contextual signals, particularly highlighting the importance of representation-level aggregation and task-specific context selection for real-world software engineering tools and workflows.

Abstract

Automated program comprehension underpins many software engineering tasks, from code summarisation to clone detection. Recent deep learning models achieve strong results but typically rely on source code alone, overlooking contextual information such as version history or structural relationships. This limits their ability to capture how code evolves and operates. We conduct an empirical study on how enriching code representations with such contextual signals affects neural model performance on key comprehension tasks. Two downstream tasks, code clone detection and code summarisation, are evaluated using SeSaMe (1,679 Java methods) and CodeSearchNet (63,259 methods). Five representative models (CodeBERT, GraphCodeBERT, CodeT5, PLBART, ASTNN) are fine-tuned under code-only and context-augmented settings. Results show that context generally improves performance: version history consistently boosts clone detection (e.g., CodeT5 +15.92% F1) and summarisation (e.g., GraphCodeBERT +5.56% METEOR), while call-graph effects vary by model and task. Combining multiple contexts yields further gains (up to +21.48% macro-F1). Human evaluation on 100 Java snippets confirms that context-augmented summaries are significantly preferred for Accuracy and Content Adequacy (p <= 0.026; |delta| up to 0.55). These findings highlight the potential of contextual signals to enhance code comprehension and open new directions for optimising contextual encoding in neural SE models.
Paper Structure (65 sections, 7 figures, 13 tables)

This paper contains 65 sections, 7 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Representations of Source Code
  4. Contextualising Code with Historical and Structural Information
  5. Evaluating Models on Program Comprehension Tasks
  6. Research Methodology
  7. Research Questions
  8. Dataset Construction.
  9. Original Datasets for Program Comprehension Tasks
  10. Context Mining Process
  11. Final Context-augmented Datasets and Statistics
  12. Evaluation Metrics.
  13. Classification Metrics:
  14. Precision, Recall, and F1-score:
  15. Macro-F1 Score:
  16. ...and 50 more sections

Figures (7)

  • Figure 1: A motivating example of using Version History to detect code clones.
  • Figure 2: Sample Java code
  • Figure 3: Sample call hierarchy context
  • Figure 4: Research Framework.
  • Figure 5: Concatenation Scenario.
  • ...and 2 more figures