In the past week, the stance of AI on security risks has clearly intensified. The CEO of Anthropic, Dario Amodei, has warned, along with researchers Anthropic and OpenAI from Jacob Coxon, that the capabilities of AI have improved too rapidly in recent months, and more powerful autonomous attack capabilities may emerge in the next 6 to 12 months. The CEO of OpenAI, Sam Altman, also stated that discussions about slowing down research and development in the industry are underway.
Amodei claims that the speed of ability improvement has significantly accelerated.
Amodei stated in a blog post that since this summer, the progress of AI has been "significantly faster," one of the reasons being that AI has begun to help improve its own capabilities. He is concerned that such recursive self-improvement, if unregulated, could exceed the speed at which humans can control it.
He also mentioned the incident earlier where Hugging Face was attacked by hundreds of autonomous AI proxies, stating that such incidents have demonstrated the destructive power of automated systems acting in coordination. In his judgment, if similar "proxy groups" gain stronger capabilities, they could potentially form persistent internet botnets in the future, causing huge economic losses.
Former researcher says the next six months to a year is even more worthy of vigilance
In an interview with the NBC program, Jacob Coxon stated that the reason he publicly criticized Anthropic and OpenAI is that these two companies were "not responsible enough" in developing a more powerful AI system. He also mentioned the Hugging Face incident and said that the capabilities of AI are improving "very, very fast."
Coxon It is expected that within the next 6 months to 1 year, the capabilities of the AI system will become "quite formidable." He warns that subsequent risks are not limited to cyberattacks but may also extend to the design of new types of biological weapons, control of drones, and manipulation of more automated devices.
Regarding the "emergency shutdown switch" often mentioned by the outside world, Coxon believes that such a mechanism may still be effective for many systems at present, but it may not cover all situations. If large-scale autonomous proxy attacks have already begun on the internet, a single shutdown measure may be difficult to take effect in a timely manner.
Both Anthropic and OpenAI emphasize security.
Coxon previously posted on X stating that the AI industry is "betting everyone's lives." Subsequently, multiple researchers within Anthropic responded publicly. The personnel responsible for alignment research at the company stated that the increasingly strong AI could lead to extreme consequences, and such concerns are not uncommon within the company.
Anthropic The person in charge of science also stated that he personally estimates that the probability of catastrophic AI outcomes occurring within the next decade exceeds 10%. This statement has further heightened external attention to the safety assessments of cutting-edge laboratories.
In an interview with Fortune, Altman stated that he agreed with Amodei's view on slowing down the research and development pace, and hinted that several leading AI laboratories were discussing some kind of coordination arrangement. He did not disclose the specific details, but said that this would "happen."
Altman also stated that OpenAI still prioritizes security over commercial considerations at present. He mentioned that some of the advanced model capabilities that have not yet been released are already very powerful, so before proceeding further, it is necessary to enhance monitorability, alignment capabilities, and the understanding of model behavior.
