Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust CAPTCHA Using Audio Illusions in the Era of Large Language Models: from Evaluation to Advances

Ziqi Ding, Yunfeng Wan, Wei Song, Yi Liu, Gelei Deng, Nan Sun, Huadong Mo, Jingling Xue, Shidong Pan, Yuekang Li

TL;DR

This work addresses the vulnerability of audio CAP-TCHAs to advanced AI by introducing AI-CAPTCHA, an evaluation framework (ACEval) for LALM- and ASR-based solvers, and a perceptual CAPTCHA (IllusionAudio) based on sine-wave speech illusions. ACEval reveals widespread security gaps in seven deployed audio CAP-TCHAs, while IllusionAudio achieves near-perfect AI resistance (0% bypass) and perfect human usability (100% first-attempt success). The approach couples automated audio generation, perceptual illusion synthesis, and irreversible conversions to exploit human-audio perception gaps that AI struggles to bridge. The findings underscore the need for AI-hard yet human-easy CAPTCHA designs and demonstrate IllusionAudio as a practical, accessible defense with broad security and usability implications for web security in the AI era.

Abstract

CAPTCHAs are widely used by websites to block bots and spam by presenting challenges that are easy for humans but difficult for automated programs to solve. To improve accessibility, audio CAPTCHAs are designed to complement visual ones. However, the robustness of audio CAPTCHAs against advanced Large Audio Language Models (LALMs) and Automatic Speech Recognition (ASR) models remains unclear. In this paper, we introduce AI-CAPTCHA, a unified framework that offers (i) an evaluation framework, ACEval, which includes advanced LALM- and ASR-based solvers, and (ii) a novel audio CAPTCHA approach, IllusionAudio, leveraging audio illusions. Through extensive evaluations of seven widely deployed audio CAPTCHAs, we show that most existing methods can be solved with high success rates by advanced LALMs and ASR models, exposing critical security weaknesses. To address these vulnerabilities, we design a new audio CAPTCHA approach, IllusionAudio, which exploits perceptual illusion cues rooted in human auditory mechanisms. Extensive experiments demonstrate that our method defeats all tested LALM- and ASR-based attacks while achieving a 100% human pass rate, significantly outperforming existing audio CAPTCHA methods.

Robust CAPTCHA Using Audio Illusions in the Era of Large Language Models: from Evaluation to Advances

TL;DR

This work addresses the vulnerability of audio CAP-TCHAs to advanced AI by introducing AI-CAPTCHA, an evaluation framework (ACEval) for LALM- and ASR-based solvers, and a perceptual CAPTCHA (IllusionAudio) based on sine-wave speech illusions. ACEval reveals widespread security gaps in seven deployed audio CAP-TCHAs, while IllusionAudio achieves near-perfect AI resistance (0% bypass) and perfect human usability (100% first-attempt success). The approach couples automated audio generation, perceptual illusion synthesis, and irreversible conversions to exploit human-audio perception gaps that AI struggles to bridge. The findings underscore the need for AI-hard yet human-easy CAPTCHA designs and demonstrate IllusionAudio as a practical, accessible defense with broad security and usability implications for web security in the AI era.

Abstract

CAPTCHAs are widely used by websites to block bots and spam by presenting challenges that are easy for humans but difficult for automated programs to solve. To improve accessibility, audio CAPTCHAs are designed to complement visual ones. However, the robustness of audio CAPTCHAs against advanced Large Audio Language Models (LALMs) and Automatic Speech Recognition (ASR) models remains unclear. In this paper, we introduce AI-CAPTCHA, a unified framework that offers (i) an evaluation framework, ACEval, which includes advanced LALM- and ASR-based solvers, and (ii) a novel audio CAPTCHA approach, IllusionAudio, leveraging audio illusions. Through extensive evaluations of seven widely deployed audio CAPTCHAs, we show that most existing methods can be solved with high success rates by advanced LALMs and ASR models, exposing critical security weaknesses. To address these vulnerabilities, we design a new audio CAPTCHA approach, IllusionAudio, which exploits perceptual illusion cues rooted in human auditory mechanisms. Extensive experiments demonstrate that our method defeats all tested LALM- and ASR-based attacks while achieving a 100% human pass rate, significantly outperforming existing audio CAPTCHA methods.
Paper Structure (24 sections, 2 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 24 sections, 2 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Audio CAP-TCHAs
  4. Large Audio Language Models
  5. Automatic Speech Recognition
  6. Audio Illusions
  7. Related Work
  8. Threat Model
  9. AI-CAPTCHA Framework
  10. ACEval: Evaluation Framework
  11. IllusionAudio: Our Method
  12. Evaluation
  13. Experiment Setup
  14. Results
  15. RQ1: Robustness of Our Method
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Architecture of content-based Audio CAP-TCHAs and rule-based Audio CAP-TCHAs.
  • Figure 2: Overview of AI-CAPTCHA.
  • Figure 3: Interface of our CAP-TCHA.
  • Figure 4: Demographic distributions of participants.