Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning Robustness of LLMs to Adversarial Typographical Errors

Esther Gan, Yiran Zhao, Liying Cheng, Yancan Mao, Anirudh Goyal, Kenji Kawaguchi, Min-Yen Kan, Michael Shieh

TL;DR

An Adversarial Typo Attack algorithm is designed that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking, which shows that LLMs are sensitive to minimal adversarial typographical changes.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning using Chain-of-Thought (CoT) prompting. However, CoT can be biased by users' instruction. In this work, we study the reasoning robustness of LLMs to typographical errors, which can naturally occur in users' queries. We design an Adversarial Typo Attack ($\texttt{ATA}$) algorithm that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking. It shows that LLMs are sensitive to minimal adversarial typographical changes. Notably, with 1 character edit, Mistral-7B-Instruct's accuracy drops from 43.7% to 38.6% on GSM8K, while with 8 character edits the performance further drops to 19.2%. To extend our evaluation to larger and closed-source LLMs, we develop the $\texttt{R$^2$ATA}$ benchmark, which assesses models' $\underline{R}$easoning $\underline{R}$obustness to $\underline{\texttt{ATA}}$. It includes adversarial typographical questions derived from three widely used reasoning datasets-GSM8K, BBH, and MMLU-by applying $\texttt{ATA}$ to open-source LLMs. $\texttt{R$^2$ATA}$ demonstrates remarkable transferability and causes notable performance drops across multiple super large and closed-source LLMs.

Reasoning Robustness of LLMs to Adversarial Typographical Errors

TL;DR

An Adversarial Typo Attack algorithm is designed that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking, which shows that LLMs are sensitive to minimal adversarial typographical changes.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning using Chain-of-Thought (CoT) prompting. However, CoT can be biased by users' instruction. In this work, we study the reasoning robustness of LLMs to typographical errors, which can naturally occur in users' queries. We design an Adversarial Typo Attack () algorithm that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking. It shows that LLMs are sensitive to minimal adversarial typographical changes. Notably, with 1 character edit, Mistral-7B-Instruct's accuracy drops from 43.7% to 38.6% on GSM8K, while with 8 character edits the performance further drops to 19.2%. To extend our evaluation to larger and closed-source LLMs, we develop the ^2 benchmark, which assesses models' easoning obustness to . It includes adversarial typographical questions derived from three widely used reasoning datasets-GSM8K, BBH, and MMLU-by applying to open-source LLMs. ^2 demonstrates remarkable transferability and causes notable performance drops across multiple super large and closed-source LLMs.

Paper Structure

This paper contains 40 sections, 2 equations, 9 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Adversarial Typo Attack (ATA)
  3. Overview
  4. Typographical Errors used in ATA
  5. Keyboard Proximity Errors.
  6. Keyboard Double-Typing Errors.
  7. Keyboard Omission Errors.
  8. Extra Whitespace Error.
  9. ATA Algorithm
  10. Task Definition.
  11. Algorithm Description.
  12. Experiment
  13. Experimental Setup
  14. Dataset.
  15. Generation of adversarial test cases.
  16. ...and 25 more sections

Figures (9)

  • Figure 1: There are two typing errors in the query: omission of a letter (year becomes yar) and duplication of a letter (has becomes haas). Consequently, in Step 1 the model wrongly wrote Regina as A, while in Step 2 the text reverses the relationship between this year's and last year's written novel. These errors in intermediate steps lead to an incorrect final answer.
  • Figure 2: ATA mainly consists of three steps: selecting a set of tokens with the highest gradients; sampling typographical errors to edit the selected tokens and generate a batch of candidates; evaluating the losses of the candidates using the model and retaining the optimal candidate for the next iteration.
  • Figure 3: Comparison of Mistral-7B responses to original (left) and adversarially edited (right) GSM8K questions. Minor typographical errors in the edited question can lead to misinterpretation and incorrect answers.
  • Figure 4: Distribution of error operations selected by ATA across the datasaets in R$^2$ATA benchmark. The predominance of whitespace errors highlights a key vulnerability in LLMs.
  • Figure 5: Distribution of edited word types in R$^2$ATA. Nouns, Verbs, and Adjectives constitute the majority of edited words.
  • ...and 4 more figures