Illustration: Mid Journey
For many years, the basic premise of artificial intelligence was simple. AI is only as good as the data it sees. Feeding them more and training them longer will improve their performance. If you feed less, it will stumble.
a new research from USC Viterbi School of Engineering suggests something even more surprising. With the right methods in place, AI models can dramatically improve their performance in areas where they have little training, far beyond what is possible with training data alone.
This method was developed by Minda Leean undergraduate student at USC Viterbi, has been working with his advisor since his freshman year. Bhaskar Krishnamachari* is a faculty fellow and systems professor in the Ming Hsieh Department of Electrical and Computer Engineering, with joint appointments in the USC Viterbi School of Engineering and the Thomas Lord Department of Computer Science in the USC School of Advanced Computing. Together, they tested GPT-5’s ability to write code in Idris, a very obscure programming language that has very little online presence compared to mainstream languages like Python. The results were amazing. By giving the AI feedback on errors and having it try again, Lee boosted the model’s success rate from a dismal 39% to 96%.
“Our AI tools can now go beyond initial training. Until maybe a year or two ago, we were saying that an AI model is only as good as the data it sees. This paper says something different.” — Professor Bhaskar Krishnamachari
A language so vague that even researchers didn’t know about it
Python, the world’s most popular programming language, has over 24 million code repositories published online, a vast library that AI models like GPT-5 learn from during training. Idris, the language Lee and Krishnamachari chose to test, has about 2,000 words. This reduces the data by a factor of approximately 10,000.
Idris’ choice was intentional and, as Krishnamachari says, a little playful too. “We were looking for a language that was so unknown that I had never heard of it,” he said. “I think we were just in my office together, Googling and trying to find some crazy language that no one had ever heard of.” They discovered Idris, a dependent functional programming language used by a small community of experts, and decided it was the perfect test case.
The point is that neither researcher could write a single line of it themselves. “Neither Minda nor I had ever written code. Frankly, we didn’t know if the code was right or wrong,” Krishnamachari admitted. That’s part of what makes this discovery so impressive. Lee was guiding the AI to learn a language that its own guides couldn’t speak.
Breakthrough: The feedback loop that changes everything
Li started by simply asking GPT-5 to solve 56 Idris coding exercises on Exercism, a popular coding exercise platform. Out of the box, the model solved only 22 cases, with a success rate of 39%, far lower than Python’s 90% and Erlang’s 74% success rates.
She then tried several approaches to improve performance, including providing documentation, error manuals, and reference guides. These helped somewhat, increasing the success rate to the low 60% range, but not dramatically.
The breakthrough came when she implemented the so-called compiler feedback loop. A compiler is software that converts code written by humans into instructions that a computer can execute. If your code is wrong, the compiler will tell you in precise and technical detail. She captured these error messages and provided direct feedback to GPT-5, requesting that they fix the specific issues identified and try again. Up to 20 times per problem.
“We thought we would see maybe a 10% increase,” said Lee, who designed and ran the experiment. “I was surprised that I was able to get to 96% by doing something seemingly simple, just keep recompiling and keep trying.”
Beyond code: why this changes everything
What Lee and Krishnamachari have built is essentially a way to unlock capabilities that have always existed but were never accessible. By designing the right kind of structured feedback, they discovered how to get much more out of their AI models than they could get from training data alone.
Krishnamachari envisions this approach being applied far outside the world of software and niche programming languages. “Imagine trying to get an AI to build a 3D model of a building,” he said. “There is something that provides feedback. This model is not structurally safe, the materials are not properly distributed, and it costs too much to build. Whatever it is, it just provides feedback with every iteration that the AI generates. What we learned from this project is that as long as we can figure out how to provide that kind of clear and correct feedback, we have the potential to significantly improve the quality of the AI output.”
He also sees applications in mathematical reasoning and legal logic, any domain with rules that are clear enough to generate objective feedback. “If you ask an AI agent to prove a theorem, it should be fairly easy to say this is wrong and here’s why, and have it investigate again.”
This research could have meaningful implications for endangered and resource-poor human languages. Former PhD student of Krishnamachari Jared Coleman works on the Owens Valley Paiute.is a Native American language with very limited written data, and is exploring whether AI can assist in translation with minimal training, reflecting the same core challenge that Lee addressed with Idris.
Once you solve one problem, solve one more.
Lee is already thinking about what’s next. Current systems essentially brute force solutions, trying and failing until something works, but what is learned from problem to problem is not retained. She wants her models to get smarter with each problem, rather than starting from scratch each time.
For Krishnamachari, the bigger picture is what AI will look like. “Part of the craziness of all this is having AI tools do tasks that we can’t do ourselves,” he said. “We’re building tools that are, in some ways, more powerful than we are.” He’s not worried about that. It excites him. He believes that AI will enable us to implement ideas that previously seemed out of reach, freeing us from drudgery and putting the responsibility of coming up with good ideas back where they belong.
After all, it all started with two people in an office Googling and wondering what would happen if they tried something a little crazy.
Published March 9, 2026
Last updated: March 9, 2026
This article may feature some AI-powered content for clarity and consistency, and to help you explore complex scientific concepts in greater depth and creative scope.
