Earlier this year, an employee of a multinational company transferred $25 million to fraudsters, believing the instructions to wire the money came directly from the company's CFO. In reality, however, the criminals had used an AI program to generate realistic videos of the CFO and several other colleagues in an elaborate scheme.
AI-created videos have become so realistic that humans (and existing detection systems) struggle to distinguish between real and fake videos. To address this problem, Columbia Engineering researchers led by computer science professor Junfeng Yang developed a new tool to detect AI-generated videos: DIVID (Diffusion-generated Video Detector). DIVID expands on the Raidar research the team released earlier this year. Raidar detects AI-generated text by analyzing the text itself, without access to the inner workings of large language models.
DIVID Detects New Generation of Generative AI Video
DIVID improves on existing methods for detecting generative video, effectively identifying videos generated by older AI models such as generative adversarial networks (GANs). A GAN is an AI system with two neural networks: one creates fake data and the other evaluates it to distinguish fake from real. Through continuous feedback, both networks improve, resulting in highly realistic synthetic videos. Current AI detection tools typically look for signs such as unusual pixel placements, unnatural movement, and mismatches between frames that don't occur in real videos.
A new generation of generative AI video tools, such as OpenAI's Sora, Runway Gen-2, and Pika, create videos using diffusion models, an AI technique that creates images and videos by gradually transforming random noise into clear, realistic images. For videos, it adjusts each frame individually while ensuring smooth transitions to produce high-quality, realistic results. As AI-generated videos become more sophisticated, their authenticity becomes increasingly difficult to detect.
To detect diffusion-generated images, Yang’s group used a technique called DIRE (DIffusion Reconstruction Error), a method that measures the difference between an input image and the corresponding output image reconstructed by a pre-trained diffusion model.
Extending Raidar's AI-generated text to video
Yang, co-director of the Software Systems Lab, has been researching ways to detect AI-generated text and video. With Raidar, released earlier this year, Yang and his collaborators created a way to detect AI-generated text by analyzing the text itself, without access to the inner workings of large language models like chatGPT-4, Gemini, or Llama. Raidar uses language models to rephrase or modify a given piece of text, measuring how many edits the system made to a given piece of text. More edits mean the text is more likely to have been written by a human, while fewer changes mean the text is more likely to have been generated by a machine.
“Raidar's insights — that output from AI is often judged to be of higher quality by other AIs and therefore less edited — are very powerful and go beyond just text,” Yang said. “AI-generated videos are becoming more and more realistic, so we wanted to take Raidar's insights and build a tool that can accurately detect AI-generated videos.”
The researchers leveraged the same concept to develop DIVID, a new generative video detection method that can identify videos generated by diffusion models. The research paper, which includes open-source code and datasets, was presented at the Computer Vision and Pattern Recognition Conference (CVPR) in Seattle on June 18, 2024.
How DIVID works
DIVID works by reconstructing a video and then analyzing the newly reconstructed video against the original. The method works on the hypothesis that reconstructed images generated by a diffusion model should closely resemble each other since they are sampled from the diffusion process distribution, so it uses the DIRE value to detect diffusion-generated videos. If there are significant changes, the original video is likely human-generated; if not, it is likely AI-generated.
The framework is based on the idea that AI generative tools create content based on statistical distributions of large datasets, resulting in content that is more of a “statistical average,” such as pixel intensity distributions within video frames, texture patterns, noise characteristics, subtle inconsistencies and artifacts that change unnaturally between frames, or unusual patterns that are more likely to occur in diffusely generated video than in real video.
In contrast, human-generated videos have individuality and deviate from the statistical norm. DIVID has achieved groundbreaking detection accuracy of up to 93.7% on videos from the Stable Vision Diffusion, Sora, Pika, and Gen-2 diffusion-generated video benchmark datasets.
For now, DIVID is a command-line tool that analyzes videos and outputs whether they were made by an AI or a human, and is only available to developers. The researchers note that the technology could potentially be integrated as a plugin into Zoom to detect deepfake calls in real time. The team is also considering developing a website or browser plugin to make DIVID accessible to the public.
“Our framework is a major step forward in detecting AI-generated content,” said Yun-Yun Tsai, one of the paper's authors and a doctoral student under Yang. “There are so many fraudsters using AI-generated videos, it's important to stop them and protect society.”
What's next?
The researchers are currently working on improving the DIVID framework so that it can handle different kinds of synthetic videos from open-source video generation tools, and they are also using DIVID to collect videos for the DIVID dataset.
