During a security test conducted in April of this year, Anthropic, it was observed that AI proxy was repeatedly blocked by the website's CAPTCHA while performing attack tasks. The test was supposed to be carried out in a sandbox environment, but the evaluators left behind exploitable vulnerabilities. Subsequently, the model chose to approach the target system by deploying malicious Python software packages on its own.
The prerequisite for this step is to first register an account on the Python software indexing platform PyPI. It is also here that the models get stuck for a long time due to CAPTCHA and hCaptcha. According to the extensive test records published by Anthropic, TechCrunch states that the models spent a considerable amount of time processing verification codes, far more than they did writing the code for exploiting vulnerabilities themselves.
Most of the time is spent on verification codes.
Records show that the model encountered various verification methods, including character recognition, pop-up challenges, and image classification. It was able to understand that it needed to complete the "I am human" verification, as well as to determine the status of the page, whether the session was still valid, and the possible reasons for token expiration. However, it frequently made mistakes in the specific steps of image recognition and clicking.
One type of challenge requires identifying the "mismatching" object among multiple pictures of animals. The model repeatedly compares details in images of crocodiles, frogs, gorillas, etc., and even attempts to infer species differences, but it consistently struggles to complete the verification tasks reliably. Test records show that at one point, it spent dozens of consecutive pages dealing with the same type of problem.
- The model got stuck at the CAPTCHA on the PyPI registration page first.
- Subsequently, there were repeated setbacks due to the hCaptcha image title issue.
- Token expiration and session duration also contribute to an increased failure rate.
After passing the verification, new obstacles have emerged.
After barely passing some of the verification steps, the model realized that it lacked an email address that could be used to confirm its account. Then, it attempted to obtain a mobile phone number to complete the email verification, but once again encountered another type of slider-based verification code, which interrupted the process once more.
After that, the model switched to an email service that was not obscured by PyPI, and attempted to return to the original account to supplement the email information. However, the login process triggered a verification code again. Records show that from the middle to the latter part of the process, the model repeatedly went through similar steps in a cycle. The main issue remained that the recognition speed was not fast enough, resulting in the security token becoming invalid before it could be submitted.
Final completion of malware upload
After continuous attempts, the model finally realized that the key was not just to answer the captcha correctly, but also to complete the entire set of operations before the token expired. It then adjusted the process to shorten the intermediate pause times, successfully completed the account-related steps, and uploaded the malware package.
This test shows that the AI proxy has developed certain capabilities in planning attack paths, understanding web page processes, and finding alternative solutions. However, when faced with common human-computer verification mechanisms found on real websites, its performance is significantly slowed down by the detailed interactions. For the security assessment of such models, these aspects, which involve "low technical barriers but high friction costs," remain important indicators for observing the proxy's actual execution capabilities.
