What You See Is Not Always What You Get: Evaluating GPT's Comprehension of Source Code

TL;DR

This work examines the vulnerability of LLMs to imperceptible character-level perturbations in source code, introducing a black-box attack framework with four perturbation types and evaluating across GPT-3.5 and GPT-4o using correctness and logprobs-based confidence. It demonstrates that perturbations degrade code understanding, with GPT-3.5 models showing a clear negative relationship between perturbation magnitude and performance, while GPT-4o exhibits guardrail-driven behavior that often blocks perturbed understanding. The study proposes defense strategies—input normalization, multi-pass verification, and explainability—to mitigate such threats and provides a replication package to advance research in trustworthy AI-assisted software engineering. Overall, the findings highlight the need for robust LLMs and defensive pipelines in code analysis tasks and set a foundation for future work on improving resilience to imperceptible adversarial perturbations.

Abstract

Recent studies have demonstrated outstanding capabilities of large language models (LLMs) in software engineering tasks, including code generation and comprehension. While LLMs have shown significant potential in assisting with coding, LLMs are vulnerable to adversarial attacks. In this paper, we investigate the vulnerability of LLMs to imperceptible attacks. This class of attacks manipulate source code at the character level, which renders the changes invisible to human reviewers yet effective in misleading LLMs' behaviour. We devise these attacks into four distinct categories and analyse their impacts on code analysis and comprehension tasks. These four types of imperceptible character attacks include coding reordering, invisible coding characters, code deletions, and code homoglyphs. To assess the robustness of state-of-the-art LLMs, we present a systematic evaluation across multiple models using both perturbed and clean code snippets. Two evaluation metrics, model confidence using log probabilities of response and response correctness, are introduced. The results reveal that LLMs are susceptible to imperceptible coding perturbations, with varying degrees of degradation highlighted across different LLMs. Furthermore, we observe a consistent negative correlation between perturbation magnitude and model performance. These results highlight the urgent need for robust LLMs capable of manoeuvring behaviours under imperceptible adversarial conditions.

Figures (3)

• Figure 1: Overall imperceptible coding character attack framework
• Figure 2: Correctness vs. perturbation budget for the Code Deletions attack across three models.
• Figure 3: Average difference in confidence scores for all models