HGAdapter: Hypergraph-based Adapters in Language Models for Code Summarization and Clone Detection

TL;DR

This work tackles the underexplored high-order correlations in code by introducing AST family, lexical, and line hyperedges, and a tokens-and-hyperedges generator to extract them. It then presents HGAdapter, a hypergraph-based adapter that interleaves with pre-trained language models to encode these correlations through an enhanced HGNN-like message-passing mechanism, while keeping PLM parameters frozen. Evaluations on code summarization (CodeSearchNet across six languages) and code clone detection (BigCloneBench) show consistent gains over full fine-tuning and standard adapters, with lexical edges delivering particularly strong contributions. The approach achieves these improvements with only a small parameter overhead, suggesting practical benefits for improving code understanding tasks in diverse PLMs.

Abstract

Pre-trained language models (PLMs) are increasingly being applied to code-related tasks. Although PLMs have achieved good results, they do not take into account potential high-order data correlations within the code. We propose three types of high-order correlations in code tokens, i.e. abstract syntax tree family correlation, lexical correlation, and line correlation. We design a tokens and hyperedges generator to capture these high-order data correlations. We improve the architecture of hypergraph neural networks and combine it with adapter tuning to propose a novel hypergraph-based adapter (HGAdapter) to fine-tune PLMs. HGAdapter can encode high-order data correlations and is allowed to be inserted into various PLMs to enhance performance. Experiments were conducted on several public datasets, including six languages of code summarization and code clone detection tasks. Our methods improved the performance of PLMs in datasets to varying degrees. Experimental results validate the introduction of high-order data correlations that contribute to improved effectiveness.

Figures (7)

• Figure 1: AST family correlation. The tokens 'a', '+', and 'b' belong to the same AST parent node, forming an addition operation, and can thus be treated as a whole unit.
• Figure 2: Lexical correlation. We use Llama BBPE-based tokenizer for tokenization. SimpleCalculator is a class name that was split into three tokens. These tokens can be viewed as a whole to extract features from their original class name.
• Figure 3: Line correlation. These tokens can be treated as a line to extract features.
• Figure 4: Overall structure of HGAdapter
• Figure 5: Results of ablation study on code summarization in CodeBERT (%)