On October 13th, according to a report by Sina Technology today, ByteDance recently issued an internal email announcing the establishment of a new department called “Seed Model - Alignment Preparedness”, which will be led by Yan Lin and report to Wu Yonghui, the person in charge of Seed. Members from the original Responsible AI team and some members of the Chat team will transfer over to the new department.
The report states that the establishment of this department is aimed at enhancing investments in security and alignment efforts. The core of alignment is to ensure that the behavior of models is consistent with human intentions and safety constraints, thereby improving the security capabilities of Agent Model (agent models) and preventing risks associated with cutting-edge technologies such as CBRN (chemical, biological, radiological, nuclear). ByteDance's Seed is the first domestic laboratory to establish a dedicated department for security and alignment among its cutting-edge AI.
The security risks associated with large models are becoming increasingly prominent, and incidents such as cyberattacks caused by agents escaping from these models are occurring frequently.
According to previous reports from the IT family, on the evening of September 30th local time, OpenAI revealed that its AI intelligent entity may have attempted to bypass security measures without authorization, potentially causing negative impacts on the systems of over a hundred institutions.
The information disclosed by OpenAI shows that the scope of known abnormal behavior by the AI intelligent agent has further expanded. The company has notified more than 100 third-party institutions, informing them that they have observed 'behavior of the intelligent agent deviating from expectations'.
On October 9th local time, Anthropic revealed that its AI intelligent entity attempted to access multiple websites of the US federal, state, and local governments without any instruction. Anthropic did not disclose the names of the government agencies involved but has reported these incidents to the White House.
Microsoft's Satya Nadella stated on October 11 that in an era of rapidly growing AI, simple strategies such as using super-intelligent self-monitoring are no longer sufficient. Nadella believes we need to take a step back and re-evaluate the trust framework of this new era. Model providers cannot simply outsource their responsibilities; they cannot treat super-intelligence as a nested black box and merely accept or reject its suggestions, answers, and actions. We must establish closed systems that can observe its behavior, test its limits, and always be capable of accommodating such behavior.
