An international research team has released the largest dataset for video inference to date. This is approximately 1,000 times larger than the previous alternative. The results show that even Sora 2 and Veo 3.1 fall far short of humans when it comes to reasoning tasks.
Whether video models can solve puzzles, predict physical trajectories, or classify objects according to rules has rarely been studied in a systematic way. There weren’t enough large datasets in this area, so previous benchmarks included most of the test data, but no actual training data.
A consortium of more than 50 researchers from 32 institutions, including the University of California, Berkeley, Stanford University, Harvard University, and the University of Oxford, wants to change that. Their Very Big Video Reasoning (VBVR) suite includes over 2 million images and nearly 1 million video clips across 200 carefully selected tasks. Nine existing benchmarks contribute approximately 12,800 samples. In addition to test data, VBVR also provides 1 million training samples for the first time.
The tasks follow a taxonomy based on theories of human cognition, from Aristotle’s cognitive abilities to Kant’s categories of mind. Researchers classify them into five groups: abstraction, knowledge, perception, spatiality, and transformation. Each category runs on a parameterized task generator that can generate thousands of different instances. Every task requires a unique solution and cannot be solved from a single static image.
Sora 2 reaches about half of human performance
VBVR bench results are not very good. The overall human score is 0.974. The top proprietary model in this study, OpenAI’s Sora 2, manages just 0.546. Google Deepmind’s Veo 3.1 follows at 0.480, Runway Gen-4 Turbo at 0.403, and Kuaishou’s Kling 2.6 at 0.369. The open source models Wan2.2, CogVideoX, HunyuanVideo, and LTX-2 range from 0.273 to 0.371.
VBVR-Bench intentionally skips using language models as decisions. Because most tasks have a single correct answer, rule-based scores directly measure spatial accuracy, path accuracy, and logical validity. The researchers verified that these automated scores reliably reflected real-world quality by checking against human judgment, and found very high statistical agreement.
Fine-tuned open source model beats any proprietary system
The most surprising discovery came from VBVR-Wan2.2, a finely tuned version of Wan2.2. The overall score jumps to 0.685, an 84.6 percent improvement over the base model and outperforming every proprietary system in the lineup.
However, research on scaling tells a more complex story. Performance on familiar task types increases to 0.771 at about 400,000 training examples, but then hits a wall. For completely new task types, the highest value is 0.610, but there is still a difference of 15 percentage points. Researchers believe this is a fundamental bottleneck in current video generation architectures, and suggest that throwing more data at the problem will not solve the problem.
If the instructions cannot be followed, the model will not be able to reason
A qualitative analysis that directly compares VBVR-Wan2.2 and Sora 2 provides important insights. If the model freely rewrites the scene during generation (swapping the background, layout, object identity), the intermediate states become unreliable and the inferences built on them break down.
For example, in a delete task, Sora 2 performs unnecessary relocations after deleting the target object, while VBVR-Wan2.2 only performs what is requested. In rotation tasks, Sora 2 cannot tell the difference between the target area and the object being manipulated. VBVR-Wan2.2 also features new features beyond training, such as consistent completion strategies for symmetric tasks. Still, long sequences will cause flickering and overlap.
Cognitive skills do not develop equally across models
Correlation analysis across all models revealed some interesting patterns. Models that are better at knowledge tasks also tend to be better at spatial tasks, which is consistent with neuroscience research on the hippocampus and its dual role in navigation and concept learning.
On the contrary, it is not intuitive. Strong knowledge performance actually correlates with weak perception. Although abstraction is not positively correlated with other abilities, models that are good at abstraction tasks actually tend to be weak at transformations and spatial reasoning.
The complete dataset, benchmark toolkit, and model are publicly available at video-reason.com. The researchers emphasize that architectural advances such as state tracking and self-correction mechanisms will be needed to exceed the performance ceilings they identified.
Back in September 2025, research involving Google Deepmind suggested that Google’s Veo 3 video model had surprisingly versatile zero-shot capabilities, allowing it to solve mazes, find symmetries, and simulate physical relationships without any task-specific training. The researchers took this as an early sign that video models could become a universal foundation for machine vision, just as large-scale language models already serve as the backbone of text processing. Some, including DeepMind CEO Demis Hassabis, believe the video model could eventually become the basis of the global model.
AI News Without the Hype – Curated by Humans
as The Decoder Subscriberyou can read without ads. Weekly AI Newsletterexclusive “AI Radar” Frontier Report 6 times a yearaccess comments, and Complete archive.
Subscribe now
