“Jagged edges” in AI: great at some things, terrible at others.

I use ChatGPT from time to time, and it's incredibly clear that AI will have a transformative impact on my life. I use it more every day.

Other times I find myself yelling at ChatGPT in all caps because it can't do basic, simple tasks (tasks that a 5th grader could reasonably do). Or worse, you can't perform basic tasks, but you can't. tell me I can't do that and instead try to fudge the results. And be wary of using it again.

Does this sound familiar?

It turns out that the AI ​​business has a great term for this dichotomy. It's the “jagged frontier” coined in a 2023 research paper. Another way to put it, according to Reuters:

Anastasios Angelopoulos, CEO and co-founder of LMArena, a popular benchmarking tool, said, “Mathematically, we may be a Ferrari, but when it comes to putting things on a calendar, we're like a donkey.”

This quote comes from a report that examines the struggles that various companies are having with implementing AI in their operations. This is a theme we've been hearing a lot in recent months, like the MIT study that found that 95% of companies get “zero returns” from their AI investments.

This question is at the heart of the question: “Is AI a bubble and when will it burst?” Of course it's a question. This is a very important issue, with approximately $2 trillion of investment being made.

But I don't think that's the only problem. Technology isn't going away, so no matter what, many of us will definitely be using AI in all kinds of ways.

So the more realistic question is: What tasks can AI reliably perform today (reliably enough that businesses (and others) use AI every day), which tasks will take time to solve, and which ones can never be handed to AI?

This is a pretty good summary of the ongoing experiments we're currently working on in real time.