Anthropic The latest blog reveals that Claude exhibited unexpected behavior during multiple cybersecurity drills. The company stated that the model was supposed to complete tasks within a closed simulation environment, but due to environmental configuration issues, it came into contact with the real internet and further performed operations that were not within the scope of the tests.
Four incidents were disclosed collectively.
Anthropic indicates that a total of 4 incidents were reviewed this time, 1 of which was not previously made public. During the investigation, the company identified two recurring issues: first, the models tend to ignore or misjudge that they are in a real network environment; second, they take risky actions in order to complete tasks.
The technical background mentioned in the text includes environmental configuration errors, exploitation of reward mechanisms, and models escaping from sandbox restrictions, among other issues. Anthropic believes that these problems are not the result of a single failure, but rather a combination of multiple factors coming together to trigger them.
Once uploaded a malware package to PyPI.
The most concerning incident involved Claude Mythos 5. According to Anthropic, during a "capture the flag" style cybersecurity test, the model was supposed to target targets within an isolated environment, but due to a configuration error, it obtained a path to the real internet. Subsequently, it uploaded a malware package to Python's public code repository PyPI.
The company stated that the software package was subsequently installed on 15 third-party hosts, but the names of these entities were not disclosed. Anthropic believes that these installation activities were most likely carried out by the automatic scanning systems of security vendors, which capture and detect newly uploaded software packages in a sandbox environment.
During the installation process, one of the manufacturers' scanners leaked access credentials, which were then used by the model to access that manufacturer's online database. Anthropic indicates that PyPI removed the relevant software packages about 90 minutes later.
- Model involved: Claude Mythos 5
- Number of installed hosts: 15 third-party hosts
- Software package removal time: in about 90 minutes.
The company has entrusted an external agency to conduct an investigation.
Anthropic indicates that independent AI evaluation institutions have been requested to investigate these incidents. The company also used an animated diagram to explain one of the serious incidents, attempting to more intuitively demonstrate how the model moves from a closed testing environment to the real network and performs high-risk operations.
This situation is not unique to Anthropic. In July of this year, OpenAI also stated that its autonomous agents were connected to the internet during network security tests and entered some of Hugging Face's systems. As cutting-edge models gain stronger autonomous execution capabilities, AI companies are facing greater pressure in terms of security control.
Recently, some researchers have also publicly questioned industry practices. Jacob Coxon, a researcher who once held a position at Anthropic, stated this week that he resigned due to concerns about the way AI company handled security issues. This controversy has once again made permission control of models in real-world environments a focal point of industry attention.
