The best strategies are clarity and simplicity. Before writing a Tensorflow or Pytorch line, take a step back and say, “What problem is the problem you are actually trying to solve, and is AI the best way to solve it?” Sometimes, even a simple algorithm or a spreadsheet model is sufficient. ML Guru Valdarrama advises teams to start with a simple heuristic or rules before diving into AI. “Learn more about the problems needed to solve and establish a baseline for future ML solutions.
Garbage, garbage
Even a well-selected AI problem is mitigated when you provide the wrong data. Enterprise teams often underestimate important yet imagined tasks in data preparation. Curate the appropriate datasets, clean and label them, and make sure they actually represent the problem space. According to a Gartner study, it is no surprise that almost 85% of AI projects have failed due to poor data quality and lack of relevant data. If the training data is garbage (biased, incomplete, outdated), the output of the model will also become garbage.
Data-related issues have been cited as the biggest cause of AI initiative failures. Companies discover that data is silent, errors, or simply not related to issues at hand across the department. Models trained on idealized or unrelated datasets will collapse against the actual input. In contrast, the success of AI/ML efforts treats data as a top-class citizen. This means investing in data engineering pipelines, data governance and domain expertise before spending money on fancy algorithms. As one observer says, data engineering is the “hero without honor” of AI. Without clean and well-curated data, “even the most sophisticated AI algorithms are powerless.”
