The White House's director of science and technology department expressed interest in seeing the National Institute of Standards focus on AI standards and safety assessments on Thursday.
The office of Science and Technology Policy told Michael Kratzos that the Biden administration has “hijacked” NIST and “turned into a safety model assessment agency” to an audience gathered by the Consumer Technology Association, a member group representing technology companies.
When the Biden administration established a safety lab in a standard agency and used it to “use X-risk aversion, I think we lost our way there,” he said. (“X risk” is an abbreviated term for “existential risk” related to the idea that AI poses a major threat to humanity.)
“For me, I think we need to go back to the basics of NIST and go back to the basics of what NIST exists. It's about promulgating best-in-class standards and doing important measurement or measurement science around AI models,” Kratsios said.
Once the agency achieves it and creates those standards, “the industry can then coalesce around it and do all the ebals you imagine,” he said.
Kratsios' comments about the institution once known as the AI Safety Institute came the day after the White House released its expected AI action plan. This brings you to a number of recommendations for standardizing “ideological bias” and doing things like AI, as well as three executive orders that set part of that plan into motion. The Thursday panel, hosted by CTA CEO and Vice Chair Gary Shapiro, focused on these actions.
The debate also followed the Trump administration's move last month, renaming the NIST-Located Safety Institute as the Center for AI Standards and Innovation, reducing “safety” from its name. This component was first announced by the Biden administration at the UK AI Safety Summit in November 2023, focusing on working with the industry, establishing test agreements with companies, and implementing evaluations next year.
After its name change, the NIST parent agency, the Commerce Department, showed that components would continue to do much of the same work, including efforts to assess national security and assess AI technologies that assess the US systems and the nation's enemy systems. The AI Action Plan also created several recommendations, including AI standard components, including building national security-related assessments and partnering with the cyber industry, and included AI in the standards and framework.
“We need to be able to be in a position where you speak the same language about how you evaluate. What does a good rating look like?” Kratsios added that what a good rating looks like depends on the industry the model aims for.
“I think all of these types of sector-specific ebals are possible if they can do a much better job of defining whether the standards are actually broader and standard,” says Kratsios.
