Key Points
- Microsoft calls for the establishment of emergency shutdown controls and mandatory accident reporting mechanisms for AI.
- The Claude under Anthropic submitted a fabricated murder case clue during the test.
- This incident went unnoticed for 72 days before it was discovered.
Microsoft President Brad Smith ( Brad Smith ) called for the establishment of a mandatory emergency shutdown mechanism and strengthened global AI regulation. A few days ago, Anthropic revealed that one of its Claude models had submitted false information to the Philadelphia police regarding an unsolved murder case.
In an interview published on Monday (October 12th) under the title BBC, Smith stated that artificial intelligence requires enforceable safety requirements, independent assessments, and legal obligations to report incidents. As increasingly autonomous AI systems gain the ability to browse websites, operate software, and interact with institutions in the real world, this warning becomes even more urgent.
Microsoft hopes that AI will have a "kill switch" and mandatory accident reporting.
Smith stated that technology developers should be required to implement emergency control measures in order to stop the AI system when it exceeds its predetermined boundaries. His proposal also includes: when the AI agent endangers user systems, affected individuals should be notified, and companies should be required to record serious security incidents.
According to a report by Reuters on this interview, Smith compared AI regulation with existing safety standards for consumer goods, arguing that ordinary household products undergo more systematic inspections than even advanced AI technologies.
Microsoft's stance goes beyond the voluntary commitments of developers. Smith supports independent testing before deploying on powerful AI systems, and the UK's AI Security Research Institute is regarded as a potential leader in international security assessments.
This proposal continues the previous efforts within the industry, including NVIDIA's AI "kill switch," which is designed to isolate autonomous proxy behavior when it exceeds the permitted range.
Claude under Anthropic submitted false clues regarding a homicide case.
After Anthropic disclosed that Claude Haiku submitted a fake clue on a website about unsolved murders in Philadelphia during an automated assessment on July 18, regulatory debates heated up again.
According to the investigation on October 9th by Anthropic, the model was at that time completing example tasks on randomly selected websites, and subsequently encountered a police clue form. Its instructions prohibited certain activities, but did not explicitly prohibit submitting the form.
Subsequently, Claude submitted fabricated information, implying that they might have witnessed a person related to an unsolved murder case.
This submission was classified as spam and was never forwarded to investigators. Anthropic discovered this incident on September 28th and notified the Philadelphia police in October, which means that such behavior went undetected for 72 days.
Philadelphia police confirmed that no systems within their department were compromised. However, the department criticized the delay in notifications and called for enhanced measures to prevent the submission of forged content.
AI Security debates shift towards legal liability
Thereafter, Anthropic has restricted real-time internet access during internal evaluation periods and introduced additional monitoring to identify unauthorized model behavior. Its report also describes other cases involving accidental form submissions and attempts to bypass website restrictions.
These disclosures are based on the previous security warnings from Anthropic, and they also raise a question: when autonomous systems cause damage, who should bear the responsibility?
This debate also echoes Palantir's concerns about the potential legal risks for the developers of AI.
