NIST launches new platform to evaluate generated AI

Image credits: Hisham Ibrahim/Getty Images

The National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce that develops and tests technology for the U.S. government, businesses, and the broader public, announced Monday that a new NIST-led announced the launch of the NIST GenAI program. AI technology, such as AI that generates text and images.

NIST GenAI releases benchmarks to help create “content authenticity” detection (i.e., deepfake checking) systems and develop software to identify sources of AI-generated false or misleading information. Encourage the newly launched NIST GenAI website and press release.

“The NIST GenAI program issues a series of challenge questions [intended] “It aims to evaluate and measure the capabilities and limitations of generative AI technologies,” the press release states. “These assessments are used to identify strategies to promote information integrity and guide the safe and responsible use of digital content.”

NIST GenAI's first project is a pilot study to build a system that can reliably tell the difference between human-generated and AI-generated media, starting with text. (Many services claim to detect deepfakes, but research and our own testing show that they are unstable at best, especially when it comes to text.) We are inviting teams from , industry, and research labs to submit one of the following “generators”: AI systems that generate content, or “discriminators,” which are systems designed to identify AI-generated content.

The research generator must provide a topic and a set of documents to generate a summary of 250 words or less, while the discriminator detects whether a particular abstract could have been written by an AI. is needed. To ensure fairness, NIST GenAI provides the data necessary to test the generator.Systems that are trained on publicly available data and those that are not[comply] NIST says it is “unacceptable” under applicable laws and regulations.

Registration for the pilot will begin on May 1st, with the first two rounds scheduled to end on August 2nd. Final results of the study are expected to be published in February 2025.

The launch of NIST GenAI and research focused on deepfakes comes as the amount of misinformation and disinformation generated by AI is rapidly increasing.

According to data from deepfake detection firm Clarity, more than 900% more deepfakes were created and published this year compared to the same period last year. Understandably, that's causing alarm.a Recent poll A YouGov survey found that 85% of Americans are concerned about misleading deepfakes circulating online.

The launch of NIST GenAI is part of NIST's response to President Joe Biden's executive order on AI, which sets rules requiring AI companies to be more transparent about how their models work, and the content generated by AI. Established a number of new standards, including labeling. .

This is also the first AI-related announcement from NIST since former OpenAI researcher Paul Cristiano was appointed to NIST's AI Safety Laboratory.

Cristiano was a controversial figure due to his “fatalist” views. He once predicted that there was a 50% chance that AI development would end. [humanity’s destruction]Critics, including scientists within NIST, fear that Cristiano will encourage the AI ​​Safety Institute to focus on “fantasy scenarios” rather than the real and immediate risks posed by AI. are doing.

NIST says NIST GenAI will provide information about the activities of the AI ​​Safety Institute.