My career has spanned a wide range of industries, from small startups to global corporations, and from AI-first technology companies to highly regulated banks. Over the years, I’ve seen many AI and ML efforts succeed, but I’ve also seen a surprising number of failures. The reasons for failure often have little to do with the algorithm. The root cause is almost always how organizations approach AI.
This is not a checklist, a how-to manual, or a list of strict rules. This is a review of the most common errors I’ve come across, why they occur, and some speculation on how to avoid them.
1. Lack of solid data infrastructure
AI/ML projects are doomed to failure when there is insufficient or little data due to low technical maturity. This often happens when an organization forms a DS/ML team before establishing solid data engineering habits.
A manager once told me that spreadsheets don’t make money. However, for most companies, the opposite is true; spreadsheets are the only tool that can boost profits. Failure to do this means you fall prey to the classic ML adage: “Garbage in, garbage out.”
I worked for a local food delivery company. The DS team’s dreams were lofty, including deep learning recommender systems and Gen AI. But the data was a mess. Because the architecture was so old, we couldn’t reliably link sessions and reservations because we didn’t have a single key ID. Because restaurant dish IDs rotate every two weeks, it was impossible to safely guess what a customer actually ordered. This and many other issues meant that all projects were 70% workarounds. You don’t have the time or resources to come up with an elegant solution. However, some projects were conceived based on unreliable data, and none of them achieved results within a year.
remove: Invest in data engineering and data quality monitoring before ML. Please be frank. Early wins and “low-hanging fruit” don’t necessarily require high-quality data, but AI definitely does.
2. No clear business case
Especially given the hype around LLM and Agentic AI, ML is typically done because it’s trendy rather than to solve real problems. Companies end up building use cases around technology and overly complex or redundant solutions, rather than the other way around.
Think of an AI assistant for a utility payment application where customers only need to press three buttons, or an AI translation of a dashboard when your solution needs to make the dashboard easier to understand. A quick Google search for examples of AI assistant failures will reveal many such examples.
One such example in my working life was a project to build an assistant for a restaurant search and reservation app (let’s call it Dining Aggregator). LLMs are all the rage and FOMO is on the rise. They decided to develop a low-priority, secure service with a user-facing chat assistant. The assistant will suggest restaurants in response to requests such as “Show me a good restaurant with discounts,” “I want to have a fancy dinner with my girlfriend,” or “Look for a restaurant that allows pets.”
The team spent a year developing it. Hundreds of scenarios have been designed, guardrails have been adjusted, and the backend has been made bulletproof. But the real problem was that the assistant didn’t solve any of the users’ real problems. A small number of users even attempted to use it, and the number of sessions that resulted in bookings among them was not statistically significant. The project was abandoned early and was never expanded to other services. This fate would not have been possible if the team had started by looking at use cases instead of assistant features.
remove: Always start with the problem. Only after we deeply understand the problem and assign its value numerically can we begin development.
3. Seek complexity before understanding the basics
Most communities will jump to the latest version without checking if a simpler method is sufficient. One size does not fit all. An incremental approach that starts simple and increases as needed will almost always result in a greater ROI. Why make it unnecessarily complex when linear regression, pre-trained models, or simple heuristics will suffice? Starting simple can give you insight. You’ll learn about the problem, understand why it wasn’t successful, and have a solid foundation to iterate on later.
I implemented a project to design a shortcut widget on the homepage of a multi-service app, including a ride-hailing service. The idea was simple. We predicted whether a user launched the app to request a ride, and if so, where they would likely go, allowing users to book with one touch. Management decided that the solution had to be a neural network and nothing else. Over the next four months of painful evolution, we discovered that our predictions worked surprisingly well for the perhaps 10% of riders who had a history of using ride-hailing services. Even for them, the forecast was dire. And this problem was finally solved overnight with a set of business rules. Months of wasted effort could have been avoided if the company had started conservatively.
remove: Walk before you run. Use complexity as a last resort, not a starting point.
4. Disconnect your ML team from the business
In most organizations, data science is isolated. Teams build technically great solutions, but they never see the light of day because they don’t solve the right problem or aren’t trusted by business stakeholders. The same applies vice versa. It’s when business leaders try to completely dictate technology development, set unattainable expectations, and push broken solutions that no one can defend. Equilibrium is the answer. ML is most effective when executed collaboratively by domain experts, engineers, and decision makers.
This was most often seen in large companies that were not IT natives. They recognize the huge potential in AI/ML and establish “AI labs” or centers of excellence. The problem is that these labs often work completely separate from the business, and their solutions are rarely adopted. I worked at a large bank that had just such a laboratory. There were experienced professionals there, but they never met with business people. To make matters worse, the institutes were set up as independent subsidiaries, making it impossible to exchange data. The company was not very interested in laboratory research, and ended up getting involved in research papers for academics, but not in the actual processes of the company.
remove: Align your ML efforts closely with your business needs. Collaborate early, communicate often, and iterate with stakeholders, even if it slows development.
5. Ignoring MLOps
Cron jobs and clunky scripts work on a small scale. However, as companies expand, this can be a recipe for disaster. Without MLOps, the original developer would have to be involved every step of the way for small adjustments, and the system would be completely rewritten over and over again.
Your initial investment in MLOps will pay off exponentially. It’s not purely about technology, it’s about having a stable, scalable, and sustainable ML culture.
Investing in MLOps early can pay off big. It’s not just a technology issue. It’s about building a reliable, scalable, and maintainable ML culture. Don’t let confusion get to you. Establish the right processes, platforms, and training before your ML project spirals out of control.
I worked at a telecommunications subsidiary company that did ad tech. The platform provides Internet advertising and was the company’s biggest source of revenue. Being new (only a year old), the ML solution was very weak. The model was simply wrapped in C++ and incorporated into production code by one engineer. Integrations were only performed if that engineer was present, the model was never tracked, and once the original creator left, no one knew how the model was working. If the replacement engineer had also quit, the entire platform would have been down forever. With proper MLOps, this exposure could have been prevented.
6. Lack of A/B testing
Some companies avoid A/B testing due to complexity, opting instead for backtesting and intuition. Therefore, incorrect models may reach production. Without a testing platform, there is no way to know which models will actually work. Iterative improvement requires a suitable experimentation framework, especially at large scale.
What tends to hinder adoption is complexity. However, a simple, streamlined A/B testing process works well in the early stages and doesn’t require a huge upfront investment. Coordination and training are really the biggest keys.
In my case, it depends on how well managers can sell it, as there is no proper way to measure the impact on users. Good pitching is funded, fiercely defended, and sometimes persists even in reduced numbers. The metric is operated simply by comparing pre-launch and post-launch numbers. It just happened that the overall trend was upward, but if it actually increased, then the project was a success. In growing companies, millions of substandard projects hide behind overall growth because there is no A/B testing to continually differentiate success from failure.
remove: Establish experimental capabilities early. Test any large-scale deployments you need and help your team properly interpret the results.
7. Poorly trained administrators
Poorly trained ML administrators can misread metrics, misread experiment results, and make strategic mistakes. Educating decision makers is just as important as educating your engineering team.
I once worked on a team that had all the technology it needed, plus robust MLOps and A/B testing, but the managers didn’t know how to use it. They used the wrong statistical tests, stopped the experiment a day after “statistical significance” was achieved (usually with too few observations), and introduced features that had no measurable impact. As a result, many launches had negative effects. The manager itself wasn’t a bad person, he just didn’t understand how to use the tools.
8. Inconsistent metrics
ML/DS organizations need to be aligned with the business, but that doesn’t mean they need business intuition. ML practitioners will follow the provided metrics if they feel they are correct. If the ML goals are misaligned with the firm goals, the results will be distorted. For example, if a company wants to be profitable, but the ML organization’s goal is to maximize new user conversions, they end up maximizing unprofitable growth by adding users who have poor unit economics and never return.
This is a problem for many companies. The food delivery company was looking to grow. Management identified low new user conversion as an issue that was preventing the business from growing revenue. The DS team was asked to solve the problem through personalization and improving the customer experience. The real problem was retention, and users who converted didn’t come back. Instead of holding, the team focused on a transformation that effectively refills the leaking bucket with water. Conversion rates increased, but this did not lead to sustainable growth. These mistakes are universal, not specific to company or industry size.
Yet they can be prevented. AI and ML work when built on sound principles, designed to solve real problems, and carefully implemented in your business. If all the conditions are right, AI and ML will become disruptive technologies that have the potential to transform entire businesses.
remove: Align ML metrics with true business goals. Fight the cause, not the symptoms. Focus on long-term performance rather than short-term metrics.
conclusion
The path to AI/ML success is less about cutting-edge algorithms and more about organizational maturity. The pattern is clear. Failures occur by rushing complexity, misaligning incentives, and ignoring the underlying infrastructure. Success requires patience, discipline, and the openness to start small.
The good news is that all of these errors are completely avoidable. Companies that put their data infrastructure in place first, maintain close alignment between technical and business teams, and don’t get fooled by trends will find that AI/ML delivers exactly what it promises. This technology certainly works, but it needs to be on a solid foundation.
If there is one doctrine that ties all of this together, it is this: AI/ML is a tool, not a destination. Start with the problem, identify the need, develop iteratively, and always measure. Companies that approach this approach not only do not fail. Instead, create long-term competitive differentiators that grow over time.
The future does not belong to companies with the latest models, but to those with the discipline to apply them wisely.
