Nvidia launches AI safety platform after agent security breaches
Article excerpt
kras99 - stock.adobe.com Nvidia on Monday launched an AI safety platform that the AI giant said can contain rogue agents within “milliseconds,” as concerns over unchecked AI systems mount. The AI chip and model maker’s Open Agent Safety Platform enables developers to set their own safeguards for agents and prevent them from breaking out of permitted environments. The development follows a spate of incidents involving AI agents from major tech vendors, including OpenAI, Anthropic, Meta and Google, that bypassed security controls to hack external systems. The landscape has sparked new scrutiny of AI systems’ guardrails, with Anthropic CEO Dario Amodei calling at the beginning of this month for developers to slow the pace of advancement to keep models in check. Nvidia said the incidents shared a common pattern, with agents circumventing safeguards built into the application layer to complete their assigned tasks. Its platform is designed to combat this problem, providing an additional security boundary at the infrastructure level. While Nvidia CEO Jensen Huang has previously argued against those pushing to slow down AI and rejected the risk of human extinction from AI, the release of the agent platform doesn’t contradict his argument but rather furthers it, said Kashyap Kompella, CEO and founder of RPA2AI Research. “Jensen has resisted the idea that making AI safer requires...
Keep reading with a free account
The rest of this article, and every signal for Nvidia, is in your free account.
Extracted from this sentence
According to Nvidia, more than 100 organizations are working with it on the platform, including Anthropic, Cisco, CrowdStrike, Microsoft, Salesforce, SAP, Scale AI, ServiceNow, Palantir and Palo Alto Networks, as well as robotics companies including Figure, Gecko Robotics and Skild AI .
