Amazon's AI assistant, Alexa, entered the AI market arguably long before other products made a name for themselves. But despite having such a huge advantage, it couldn't maintain its momentum. Amazon lost its game at some point, and other companies like OpenAI, Microsoft, and Google came in and dominated the field. There are many rumors floating around the market as to why Amazon lost the battle to win, but here is one story told by a former senior machine learning scientist at Alexa AI.
Alexa AI scientist Mihail Eric posted an article about why Alexa lost at X.
Why Alexa lost: Data mislabeling
According to Eric, Alexa AI was full of technical and bureaucratic problems. Alexa put a lot of effort into protecting customer data by putting guardrails in place to prevent it from being leaked or accessed, he said. This was a very important practice, but he added that it resulted in the internal infrastructure being very difficult for developers to deal with.
According to him, even if they needed to access the internal data for analysis or experimentation, it would take weeks. Not only is the data poorly annotated, but the documentation is not well maintained and is either non-existent or out of date. Moreover, he said, the experiments have to be performed in resource-limited computing environments. He gave the example of training a Transformer model when only a CPU is available.
He went on to tell us a story about when his team's analysis revealed that the annotation scheme for a small subset of speech data was completely wrong. What his team was trying to prove was that the resulting data labels were inaccurate – that for months, their in-house annotation team had been mislabeling thousands of data points every day. When they tried to change the annotation taxonomy, they found that even small changes required a lot more work than they expected.
He had to get the product manager on board, get that manager's buy-in, submit a preliminary change request, and get it approved (a process that took months end-to-end).
According to Eric, the reason this stalled was that there was no incentive for the product manager to fix the problem, as there was no promotion reason. The only reason the product manager gave for fixing this problem was that it was “scientifically correct and could lead to a better model for other teams.” Since there was no incentive, no action was taken.
Why Alexa lost: A fragmented organizational structure
He then talked about how Alexa's organizational structure was decentralized by design, with multiple small teams spread across geographies working on sometimes the same problems. Teams, he said, were in a rush to get work done to avoid being reorganized and absorbed by competing teams.
The result, scientists say, has been an organisation plagued by hostile middle managers who have little interest in working together for the greater good of Alexa as a whole and only want to maintain their own sphere of influence.
He recounted a story about when he and another team were coordinating a project to scale out the training of large Transformer models, which, if done correctly, could have been the origin of Amazon ChatGPT (long before ChatGPT was released).
He said, “Our Alexa team met with our internal cloud team, who had begun a similar effort on their own. The goal was to find a way to collaborate on this training infrastructure, but over the course of several weeks there were a lot of half-baked promises that never came to fruition. In the end, our team was doing its own thing, and our sister teams were doing their own thing. With no common ground, there was duplication of effort. With no sharing of data, infrastructure, or lessons learned, the quality of the models produced inevitably suffered.”
Why Alexa lost: Product and science mismatch
He tweeted, “Alexa is relentlessly customer-centric, which I think is admirable and a principle every company should live by. At Alexa, this meant that all engineering & science efforts had to be aligned with some downstream product. It certainly created tension for the team, as we had to make experimental bets on the future of the platform. These bets couldn't be baked into product in a normal quarter without hacks and shortcuts, as was expected. So we constantly had to justify our existence to senior management and align projects with metrics that were perceived as more customer-centric.”
He then gives an example: “For example, in one of our projects to build an open-domain chat system, the success metric imposed by senior management (i.e., a single integer value representing overall conversation quality) had no scientific basis and was nearly impossible to achieve. This led to product vs. science conflicts in weekly meetings to track project progress, managers changing every few months, and eventually the effort being abandoned.”
