CAISI Enters Security Testing Agreements With Google, Microsoft, and xAI

The Center for AI Standards and Innovation (CAISI), a unit of the National Institute of Standards and Technology (NIST), announced on Tuesday new security testing agreements with Google DeepMind, Microsoft, and xAI.

The agreements build on previously announced research and testing agreements with Anthropic and OpenAI, which CAISI announced in August 2024. CAISI said it has since renegotiated these agreements to reflect Trump administration priorities.

“Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications,” said Chris Fall, the newly appointed director of CAISI, in a statement. “These expanded industry collaborations help us scale our work in the public interest at a critical moment.”

CAISI is the federal entity responsible for evaluating frontier AI models.

The testing agreements allow the federal government to evaluate the companies’ AI models before they are released to the public, as well as conduct post-deployment assessments and research. CAISI said it has completed over 40 such evaluations to date, including on “state-of-the-art models that remain unreleased.”

In a press release, CAISI said these collaborations “support information-sharing, driving voluntary product improvements and ensuring a clear understanding in government of AI capabilities and the state of international AI competition.”

The news comes as the federal government deepens its engagement with leading AI developers, including recent Pentagon deals with major tech firms.

At the same time, officials have raised concerns about the potential misuse of increasingly capable AI models. Anthropic confirmed it has been in discussions with U.S. government officials regarding its Claude Mythos Preview model, which includes advanced hacking capabilities.