- New Runway study asks volunteers to identify AI videos
- Clips were correctly identified only 57.1% of the time
- Video was produced with the new Runway Gen-4.5 model
When it comes to the veracity of AI-generated videos, we seem to be looking through the looking glass. In a new test conducted by AI video company Runway with more than 1,000 participants, volunteers were able to correctly distinguish between AI and real videos only 57.1% of the time.
This is a worryingly low number, considering that a random guess would land you at around 50%. Even Anastasis Germanidis, Runaway’s co-founder and chief technology officer, admitted that he failed “pretty much” at this task (via The Information).
If a Runway team member is struggling, the rest have little chance. The fake and real clips were set to the same resolution and length, and volunteers had 10 seconds to decide which ones showed real people in the real world and which ones were created by AI (reflecting how fast people scroll on social media).
The test was commissioned to mark the wider rollout of Runway’s new Gen-4.5 model, which the company promises will offer “unprecedented visual fidelity and creative control” and deliver “cinematic and highly realistic” results.
“Think more critically”
watch on
Participants tested performed best when the human face, hands, or movements were involved, with accuracy rates for these clips ranging from 58 to 65 percent. Germanidis said this could be because the “creepy” elements are most prominent in these videos, even if the AI video itself is of the same quality as the real video.
Germanidis encourages people to “think more critically” when weighing what they see online. “At this point we have crossed the threshold and it is difficult to distinguish it from what is generated from real video,” he told The Information.
Runway is working on ways to reliably watermark AI output, and its videos already include metadata by default to indicate they were generated by AI. Naturally, governments are eager to implement this feature, as it completely erases the line between what is real and what is computer-generated.
However, Runway Gen-4.5 is not yet perfect. Runway says they can still struggle with disappearing objects, causal reasoning (a door might open before you pull the handle), and success bias (such as a poorly aimed kick going into the net, such as in a soccer scene).
Follow TechRadar on Google News and Add us as a preferred source Get expert news, reviews, and opinions in your feed. Be sure to click the follow button!
Of course you can also do Follow TechRadar on TikTok Check out news, reviews, and unboxings in video format and stay updated regularly. whatsapp Too.
The perfect business laptop for every budget
