VIPER Strike: Defeating Visual Reasoning CAPTCHAs via Structured Vision-Language Inference

TL;DR

ViPer tackles the challenging Visual Reasoning CAPTCHAs by decoupling high-precision visual perception from adaptive, prompt-guided LLM reasoning, enabling robust cross-platform solving. The framework uses a structured perception backbone to inventory objects with compound labels and aligns this with NL queries through semantic extraction, integration, and relative-position reasoning, culminating in an image-space coordinate via a dynamic prompt. Across six real-world VRC platforms, ViPer achieves up to 93.2% accuracy and outperforms prior solvers (GraphNet, Oedipus, Holistic), with robustness across multiple LLM backbones. A lightweight defense, Template-Space Randomization, demonstrates reduced solver success, signaling a path toward human-solvable but machine-resistant VRCs.

Abstract

Visual Reasoning CAPTCHAs (VRCs) combine visual scenes with natural-language queries that demand compositional inference over objects, attributes, and spatial relations. They are increasingly deployed as a primary defense against automated bots. Existing solvers fall into two paradigms: vision-centric, which rely on template-specific detectors but fail on novel layouts, and reasoning-centric, which leverage LLMs but struggle with fine-grained visual perception. Both lack the generality needed to handle heterogeneous VRC deployments. We present ViPer, a unified attack framework that integrates structured multi-object visual perception with adaptive LLM-based reasoning. ViPer parses visual layouts, grounds attributes to question semantics, and infers target coordinates within a modular pipeline. Evaluated on six major VRC providers (VTT, Geetest, NetEase, Dingxiang, Shumei, Xiaodun), ViPer achieves up to 93.2% success, approaching human-level performance across multiple benchmarks. Compared to prior solvers, GraphNet (83.2%), Oedipus (65.8%), and the Holistic approach (89.5%), ViPer consistently outperforms all baselines. The framework further maintains robustness across alternative LLM backbones (GPT, Grok, DeepSeek, Kimi), sustaining accuracy above 90%. To anticipate defense, we further introduce Template-Space Randomization (TSR), a lightweight strategy that perturbs linguistic templates without altering task semantics. TSR measurably reduces solver (i.e., attacker) performance. Our proposed design suggests directions for human-solvable but machine-resistant CAPTCHAs.

Figures (13)

• Figure 1: Example of a Visual Reasoning CAPTCHA (VRC). A solver ① identifies the yellow lowercase letter (perception); ② matches it with its corresponding uppercase form (reasoning), producing the correct answer 'Q'.
• Figure 2: ViPer end-to-end pipeline.Exterior ingests image $I$ and instruction $q$. Viewer conducts structured perception: a multi-object detector yields labeled boxes $\mathcal{V}$; QIE parses $q$ into attribute slots; the Integrator aligns slots with $\mathcal{V}$ to obtain candidate set $V'$; if $q$ includes a spatial phrase, RPIE resolves references via geometric projection (§\ref{['sec:viewer-details']}- §\ref{['sec:rpie']}). Policymaker composes a task-conditioned prompt from $q$ and $V'$, queries an LLM, and outputs a coordinate $(x,y)$ (§\ref{['sec:policymaker']}).
• Figure 3: Semantic extraction and alignment. QIE parses the query into slots; the Integrator matches them with detector outputs to yield candidate set $V'$.
• Figure 4: Task-conditioned prompts for Policymaker (§\ref{['sec:policymaker']}). Spatial prompt that names the relation and reference; Comparative prompts define an attribute and target class; Direct-attribute prompts bind shape and color.
• Figure 5: Schematic of ablation variants. R1 relies solely on native multimodal vision and zero-shot reasoning. R2 augments the prompt with detector outputs but omits semantic filtering and relative-position inference.