AI has the potential to transform business, but real-world failures at companies like Air Canada, Zillow, Samsung, CNET, and IBM show how quickly things can go wrong.
adobe stock
AI is already reshaping businesses, but it’s also exposing some very expensive weaknesses.
Despite the excitement around generative AI, automation, and intelligent decision-making, the value of technology lies in the strategy, governance, and human judgment behind it. Without these, AI can undermine customer trust, leak sensitive data, create legal issues, and turn small mistakes into very expensive bills.
Here are five real-world mistakes made by AI and the lessons every business leader should learn from them.
Air Canada: Chatbot illusions
In 2024, a Canadian court ordered Air Canada to pay compensation after a chatbot embedded in its online reservation system gave the airline the illusion of a fictitious discount. The bot reportedly provided incorrect advice about fare discounts to a passenger attending his grandmother’s funeral, assuring the passenger that they could pay the fare in full and claim the discount retroactively.
This turned out to be against company policy, and Air Canada refused to honor the discount. However, the court ruled that the passenger should pay $812.02 due to the chatbot’s mistake.
Gabor Lukacs, president of Air Passenger Rights, summed up the lesson: “When you hand over part of your business to AI, you are responsible for what the AI does.”
Zillow: Machine learning miscalculations
When real estate services specialist Zillow used machine learning to build a tool to automatically buy homes and make money, the results weren’t what they expected. Its algorithmic model is designed to find optimal buying and selling prices to maximize trading profits, but it turns out that it cannot accurately predict the chaotic movements of the real estate market. This resulted in an overpayment and a loss of $500 million.
Eventually, the entire division shut down and Zillow canceled the project as a costly lesson learned. AI mistakes can escalate quickly, and a small miscalculation or “rounding error” can quickly turn into a major disaster if left unchecked.
Samsung: Governance failure
Samsung was forced to crack down on its employees’ use of generative AI tools after discovering that employees were uploading sensitive company information. Anything entered into a cloud-based AI chatbot like ChatGPT can be viewed by a human operator and used to further train the AI. Simply put, what happens to this data is outside of the company’s control. In the case of Samsung, this highlighted a serious lack of governance around how AI is used. Unfortunately, this is still the case for many companies today. Shadow AI is becoming more prevalent as employees use unapproved tools because they find them fast and useful, while unclear guidelines and policies mean employees don’t know what AI should or shouldn’t be used for. Make sure your company is not one of them.
CNET: Inadequate human oversight
Trust between readers and publishers is critical in journalism, and technology news outlet CNET cracked that relationship with an AI-generated article. Complaints about inaccuracies skyrocketed after we started including AI descriptions along with features and reviews. A subsequent investigation found errors in 41 of the 77 AI-generated articles. In addition to the loss of trust, the fact that human writers had to spend considerable time publishing lengthy revisions undoubtedly added to the damage, but the total amount is unknown. The lesson here is that robust processes need to be in place to ensure that AI content is subject to human review and oversight.
IBM: The mismatch between hype and reality
IBM’s Watson Health platform serves as a warning to those who tend to get overly excited about unproven possibilities. IBM spent billions of dollars building and marketing healthcare AI, and while expectations were high, the results didn’t really add up. The technology has produced inconsistent results, adoption has stalled, and trust has eroded. IBM ultimately sold Watson Health, which at one time had 7,000 employees, and learned that it was probably a good idea to wait for results to be validated before declaring a product market-ready.
Despite accelerating efforts around AI regulations, guidelines, and responsible practices, I believe that more companies will make significant AI-related mistakes in the near future. Companies fail sometimes because they tackle AI too quickly for fear of being left behind, and sometimes because their organizations are not aligned from top to bottom on AI governance and oversight issues.
Understanding common causes of mistakes and learning from those who have made them before can help companies be better prepared to avoid mistakes or, when they can’t, minimize collateral damage.
