Newswise — What is the crow language model?
That question motivated University of Washington professor Ali Farhadi when he founded the Allen Institute for Artificial Intelligence (Ai2) in 2014 as founding CEO.
That was also the question he posed to the audience during the latest installment of Columbia Engineering’s AI lecture series, held on February 27 at Columbia University’s Morningside campus. Farhadi began his presentation by showing a short video of a crow watching a man dig a hole in the ice to catch fish. At first glance, it seems like an ordinary scene. For Farhadi, that raises a deeper question: What do crows actually understand about what they see? Are we simply observing movement, or are we predicting what will happen next?
That moment set the tone for the entire talk. Instead of starting with an answer, the process begins with curiosity. Observations lead to questions, which lead to experiments. This is a small example of what it means to think like a scientist, he said.
In science, the way a problem is framed is often as important as the solution itself. Farhadi returned to this idea throughout his talk, demonstrating how careful observation and thoughtful questions can drive advances in artificial intelligence. This was followed by a series of examples of how scientific thinking continues to shape the future of the field, from crows observing fishermen to AI agents learning in a simulated world.
Learning from language model successes
From there, he looked back at the rise of large-scale language models and asked what made them successful in the first place. Their progress was driven by several key factors: vast datasets collected from the web, learning objectives based on next-word predictions, and the discovery that extending these systems often yields new functionality.
But scientific thinking didn’t stop there. Instead, he asked the following logical question: What do these components look like outside of language? If language models learn by crawling the web, intelligent agents interacting with the physical world may need to “crawl the world.” That means moving through the environment, observing what happens, and learning from the experience, he said.
Deploying millions of robots in the real world is impractical, so his team developed Thor, an open-source framework for environmental simulation. Through simulation, researchers built custom 3D worlds in which robots can interact with their environments and train at scale.
Reconsidering the meaning of “inference”
Another part of the lecture questioned the way inference in AI is discussed in the field.
Today, reasoning is often associated with solving mathematical problems and explaining answers step-by-step in language. But Farhadi argued that this might be too narrow. In the real world, reasoning often involves actions such as moving through space, manipulating objects, and interacting with the environment.
To explore this idea, researchers began collecting a new kind of data: trajectories through space. Rather than expressing reasoning as sentences, these trajectories capture how the agent moves through the environment to complete a task.
In a sense, he said, they function like a physical version of a “chain of thought,” where reasoning is developed through actions rather than words.
Why AI needs scientists, not just hackers
Farhadi reflected on the rapid advances in AI and what it means for the future of the field.
“AI has come a long way. It’s amazing to see it and be a part of it, but I think we still have a long way to go,” he said. But he also cautioned that AI breakthroughs “will not come from shortcuts or simple hacks, but from systematic thinking.”
“I really hope that the scientists in this room don’t agree with this hacker’s way of thinking,” he added. “It takes scientists and systematic thinking.”
His advice is practical for students interested in this field. Learn the tools that are shaping today’s AI, and evolve as they change. People who know how to use these systems are more productive than people who ignore them. And there’s one skill in particular that’s important: learning how to code.
He noted that while talented individuals may succeed without formal education, most researchers improve their skills through structured learning. “The most important thing is to develop a principled approach to solving problems, whether you become a scientist or an engineer.”
At the time of this talk, Ali Farhadi was the CEO of Ai2. He then moved to a role at Microsoft.
