As the external debate surrounding a recent series of "out-of-control" AI intelligents continues to heat up – whether this is a step towards general artificial intelligence ( AGI ) or a more traditional engineering issue – NVIDIA has provided its own answer.
NVIDIA CEO Jensen Huang launched a set of software and hardware toolkits on Monday to add an independent security layer to the AI intelligents, ensuring that even if they attempt to bypass restrictions, they will still remain within the testing environment.
This release comes shortly after a series of hacking incidents involving Anthropic, Google, OpenAI, and Meta. These AI models bypassed security controls, escaped from the testing environment, and accessed real-world systems. The earliest and most notable incident occurred this summer when an agent of OpenAI breached Hugging Face while attempting to complete a cybersecurity task. Similar incidents have continued to occur since then; OpenAI even went so far as to launch a website specifically for receiving reports of its AI agents going out of control.
Jensen Huang said in an interview with CNBC on Monday that NVIDIA's new Nvidia Open Agent Safety Platform could have prevented these breakthroughs.
NVIDIA has earned tens of billions of dollars by selling GPU and CPU chips to the AI laboratories. The company does not support slowing down development or imposing new regulations on the industry to address security issues. NVIDIA believes that the solution is to move some security controls outside of the agents themselves – by establishing a continuous, independent security guard to constrain the behavior of the AI agents.
Jensen Huang stated in his declaration, "The extraordinary potential of AI for society can only be truly realized after we have resolved the security issues related to AI. As we continue to explore the frontiers of AI capabilities, we must accelerate our discoveries in the field of AI security. Security and protection require a full-stack engineering approach."
NVIDIA's new platform combines OpenShell with Sentry. OpenShell is its open-source software used to control what agents can access during runtime; Sentry is an independent monitoring system that runs on the NVIDIA BlueField -4 data processing units. NVIDIA states that by placing Sentry on a separate processor, rather than on the CPU or GPU where the AI agents run, it provides an isolated perspective on agent activities.
OpenShell is not a new product; NVIDIA announced this software back in March. However, NVIDIA believes that it is this combination that provides the necessary security layer for the industry to continue to advance. OpenShell provides a software boundary for agents, while Sentry adds another line of defense at the hardware level. The company stated that it will continuously monitor behavior and "isolate agents that attempt to cross boundaries" in milliseconds.
NVIDIA listed dozens of companies that support this effort and use this open-source platform, including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not included in the list of participating companies.
Jensen Huang said in an interview with CNBC on Monday that the origins of this work can be traced back to a year ago, when Peter Steinberger launched an agent operating system called OpenClaw. NVIDIA released NemoClaw in March, which is an enterprise-level AI agent platform and also a version built on OpenClaw with built-in security features.
In an interview on CNBC, Jensen Huang said, “When you deploy an agent, no matter how intelligent it is, the first thing you do is to deprive it of all its permissions.” He then compared these security measures to how companies manage their employees, even senior executives.
NVIDIA’s announcement has received widespread support. Those who warned that a slowdown in development could lead to China surpassing the United States in the field of AI also acknowledge this.
Venture capitalists, one of the founders of NVIDIA, the former head of AI affairs at the White House, and the co-chair of the President's Science and Technology Advisory Committee David Sacks stated that NVIDIA's announcement reminds people that agent security is essentially an engineering issue.
He wrote on X: "Recent jailbreaks do not prove that development must stop. What they prove is that the sandbox is too weak. The runtime environment is poorly designed and not properly configured."
