Openai's performance chart for the GPT-5 launch video is so confused that I think GPT-5 itself probably created them, and the company's attempted fixes raise even more questions

What to do with Openai's latest GPT-5 chatbot? Let's say the user's receptions are mixed well for Openai head Honcho Sam Altman to apologise and post to X. etc. But one thing we can say for certain is that the launch video charts were the strange confusion Openai has tried to sort out and mix since then.

Most obviously, the older model shown on the GPT-5's claimed SWE bench performance and release date failed badly. The chart showed accuracy figures of 74.9% for ChatGPT 5, 69.1% for OpenAI O3 and 30.8% for GPT-4O.

chatgpt-5 chart — Before and after. (Image credit: Openai)

The problem is that if the bar graph height is exactly the same for the latter two and is actually slightly better than the Openai O3, it gives the overall impression of the complete advantage of GPT-5.

It's a basic enough mistake, so you have to wonder whether Openai used it or whether GPT-5 itself could not bother to create charts and prove them. Later in the video, we see a different graph showing GPT-5's deception rate, not a bit ironic. This chart shows bar graphs for GPT-5 and Openai O3.

GPT-5 gets 50% “coding cheating” and Openai O3 is 47.4%. However, the bars on Openai O3 are rendered about three times higher than that of the GPT-5. Now you can make some complicated arguments to make the lower deception rates better and therefore higher the bars in Openai O3.

GPT-5

These charts are very bad, the GPT-5 must have made them. (Image credit: Openai)

Apart from the fact that this approach still does not explain the big discrepancies in bar height, the problem is that it also shows statistics for “Charxiv missing images” on the same slide. Here, the bars are accurately proportional to the outcome. 9% of GPT-5 is a small percentage of Openai O3's 86.7% high.

Another unstable chart cooked by AI? Some sort of subtle satire? Lazy and sloppy work? A conspiracy where mere incompetence is sufficient is probably unfair, as normal sayings probably apply. But that certainly means a squared level of self-satisfaction with an overall sense of qualifications, rigour and accountability surrounding the entire AI industry.

Openai has been around since then Updated charts have been posted on our website. The new deception rate chart certainly suggests that a mere mistake has been made. Revised statistics show that the coding deception rate for GPT-5 is 16.5%.

However, although the bar height on the SWE Bench Chart has also been revised, Openai added a further disclaimer, pointing out that the figure was achieved using 477 tasks in the SWE Bench Suite instead of 500 yen.

When compared directly to Anthropic's Claude Opus 4.1, which achieved a 74.5% score on SWE bench verification, the GPT-5 appears to be slightly better at 74.9%. But where are the issues with these 23 missing? what are they? pic.twitter.com/f8bsznxlzt August 11, 2025

This led some observers to question whether some inconvenient tasks were ruled out to allow GPT-5 to reach 74.9%, thus slightly surpassing the 74.5% score acquired by Anthropic's Claude Opus 4.1 model. Certainly, this raised the brows of someone other than Elon Musk.

Meanwhile, Openai's YouTube channel still has original launch videos with messed up charts. No matter what's going on, exactly that is all pretty unseemly. At best, it is the existence of an organization that appears to produce artificial “intelligence.” In the worst case, if you manage risky AI and all of our safety depends on these people, it's totally unsettling.