High-tech companies are fiercely competing to build the best AI coding tools. For Xai, it seems that humanity is Beat's top rival.
Elon Musk's AI company has used contractors to train Grok on coding tasks with the aim of topping popular AI leaderboards, explicitly telling them it hopes to outweigh Anthropic's Claude 3.7 Sonnet tool, document obtained by Business Insider Show.
Contractors hired through Scale AI's Outlier platform have been assigned the project to the “Hillclimb” Grok rankings at WebDev Arena. Webdevanirea is an influential leaderboard that attacks AI models on each other in web development challenges, with users voting for winners.
According to one contractor who worked on the project, “I want to turn my task-in model into a #1 model to #1 model,” he reads an active one-scale AI onboarding document in early July. The contractor has “extended Sonnet 3.7” references to the Claude model of humanity by generating and improving front-end code for the user interface prompt.
Xai did not reply to the BI request for comments.
Without universally agreed criteria, leaderboard rankings and benchmark scores have become the unofficial scoreboard for the AI industry.
For labs like Openai and Humanity, topping these rankings can help you attract funds, new customers, lucrative contracts and media attention.
With multiple models, Anthropic's Claude is considered one of the leading players in AI coding, and is consistently ranked near the top of many leaderboards, often alongside Google and Openai.
Humanity co-founder Ben Mann said last month on a No Priors podcast that other companies had declared to match Claude's coding capabilities, shocking that other models hadn't kept up. Competitors like Meta use human coding tools internally, BI previously reported.
The scale AI dashboard and project instructions did not specify the version of Grok the project is training for, but it was in use just a few days before its latest model, the Grok 4, was released on July 9th.
Related Stories
Business Insider tells the innovative stories you want to know
Business Insider tells the innovative stories you want to know
On Tuesday, Lmarena ranked Grok 4 at No. 12 for web development. The human model is ranked in the first, third and fourth joints.
The day after the release of Grok 4, Musk was posted to X, revising the code by referring to popular AI-assisted developer tools, claiming that the new model is “more effective than Cursor.”
You can cut and paste the entire source code file into the query entry box at https://t.co/eqiifyhflo @grok 4 will fix it for you!
This is everyone @xai I will. It works better than a cursor.
– Elon Musk (@elonmusk) July 10, 2025
In a comment to BI, Scale Ai said that training directly on the test set would not overfit the model. The company told BI it would not copy or reuse public benchmark data for large-scale language model training, and told BI it was engaged in a “standard data generation project using public signals to close known performance gaps.”
Lmarena CEO Anastasios Angelopoulos told BI that hiring contractors to help AI models climb public leaderboards is a standard industry practice, although he was unaware of projects of a particular size.
“This is part of the standard workflow for model training. We need to collect data to improve the model,” Angelopoulos said, adding that it is “not just web development, but any benchmark.”
Leaderboard domination race
The focus of the industry on AI leaderboards is competitive and not always fair.
Sara Hooker, head of Cohere Labs, is one of the authors of The Leaderboard Illusion, a paper published by researchers at universities, including MIT and Stanford, who told BI that “if leaderboards are important to the entire ecosystem, incentives will align to align.”
In April, after Meta's Llama 4 model came in second in the LM Arena, the developers noticed that the model variant Meta used for the public benchmark was different from the published version. This sparked accusations from AI researchers that Meta was betting the leaderboard.
Meta denied the claim, saying that the variant in question was experimental and that it is a standard practice to evaluate multiple versions of the model.
Xai's project with Scale AI asked contractors to help “Hillclimb” with Lmarena rankings, but there is no evidence that they are betting the leaderboard.
Leaderboard advantages do not always lead to real-world capabilities. Another author of the paper, Shivalika Singh, told BI that “good things in the arena generally not a good performance,” and that he guarantees strong results on other benchmarks.
Overall, Grok 4 ranked in the top three in the core categories of math, coding, and “hard prompts”.
However, early data from the new crowdsourced leaderboard and Lmarena rival Yupp showed that the Grok 4 ranked 66th out of over 100 models, highlighting differences between leaderboards.
Nate Jones, an AI strategist and product leader with a widely read newsletter, said he noticed that Groke's actual capabilities often fall behind leaderboard hype.
“The Grok 4 crushed some flashy benchmarks, but when the rubber came across the road in my test this week, the Grok 4 stumbled violently,” he wrote to his subjew on Monday. “The moment we set leaderboard domination as our goal, we run the risk of creating trivial exercises and outstanding models of flounder when faced with reality.”
