Breaking reCAPTCHAv2

TL;DR

This work examines the efficacy of employing advanced machine learning methods to solve captchas from Google's reCAPTCHAv2 system, and finds evidence that reCAPTCHAv2 is heavily based on cookie and browser history data when evaluating whether a user is human or not.

Abstract

Our work examines the efficacy of employing advanced machine learning methods to solve captchas from Google's reCAPTCHAv2 system. We evaluate the effectiveness of automated systems in solving captchas by utilizing advanced YOLO models for image segmentation and classification. Our main result is that we can solve 100% of the captchas, while previous work only solved 68-71%. Furthermore, our findings suggest that there is no significant difference in the number of challenges humans and bots must solve to pass the captchas in reCAPTCHAv2. This implies that current AI technologies can exploit advanced image-based captchas. We also look under the hood of reCAPTCHAv2, and find evidence that reCAPTCHAv2 is heavily based on cookie and browser history data when evaluating whether a user is human or not. The code is provided alongside this paper.

Figures (6)

• Figure 1: Examples of the three different captcha type challenges used by Google's reCAPTCHAv2. Each type presents a unique challenge for users to solve to determine whether the user is a bot or not.
• Figure 2: Normalized confusion matrix of the fine-tuned YOLOv8 model evaluated on the 13 classes seen in captcha challenges. The top 1 accuracy is 82.4% while the top 5 accuracy is 99.5%. The matrix highlights the model's ability to correctly classify various objects such as bicycles, bridges, buses, cars, and more, with values indicating the proportion of correct predictions. For example, bicycles were correctly identified with an accuracy of 89%, while bridges and buses had an accuracy of 84% and 97%, respectively. The matrix reveals the strengths and weaknesses of the model in different classes, showing high precision in certain categories, such as hydrants (100%), and notable confusion in others, such as varied performance in the identification of cars, illustrating the challenges of distinguishing between closely related objects.
• Figure 3: Comparative analysis of captcha-solving challenges with and without the use of a VPN. The upper graph (a) shows the challenges without VPN, where the bot is flagged after the 19th run. The lower graph (b) demonstrates consistent performance over 100 runs with VPN, avoiding bot detection and subsequent challenge escalation.
• Figure 4: Comparison of different mouse movement strategies in captcha solving. From top to bottom: (a) without mouse movement, (b) with straight line movement, and (c) with Bézier curve movement, illustrating the progressive performance improvement.
• Figure 5: Comparative analysis of captcha-solving challenges with and without browser history and cookies from a browser session with a logged-in Google account. The left graph (a) displays the number of challenges in the absence of cookies and history, while the right graph (b) shows the number of challenges with cookies and history present, indicating the impact of user data on captcha challenge complexity.