author: Kevin Stumpf, Co-Founder and CTO
When we started rolling out Uber’s machine learning platform Michelangelo in 2015, we noticed an interesting pattern. 80% of ML models launched on the platform operational Machine learning use cases directly impact the end-user experience (Uber riders and drivers). Only 20% analytical machine learning Use cases to enhance analytical decision making.
The ratio of operational to analytical ML we observed was accurate. Opposition When it comes to how most other companies are actually applying ML, analytical ML has been king. In hindsight, it’s no big surprise that Uber adopted operational ML at scale. Michelangelo made it very easy to implement operational ML, and the company had a rich set of high-impact use cases. Seven years later, Uber’s reliance on operational ML has only grown. Without this, hundreds of millions of dollars would be lost in uneconomic ride fares, poor arrival time predictions, and fraud. In other words, without operational ML, your company will grind to a halt.
Operational ML is the key to Uber’s success, and for a long time it seemed like something only the tech giant could achieve. But the good news is that a lot has changed in the past seven years. There are new technologies and trends that are enabling any company to switch from primarily using analytical ML to using operational ML, and we offer some tips for those looking to do so. Let’s dive in.
Operational ML and analytical ML
operational machine learning Applications use ML models to autonomously and continuously make business-impacting decisions in real time. These applications are mission-critical and run “online” in a production environment on a company’s production stack.
Common examples include recommendation systems, search rankings, dynamic pricing, fraud detection, and loan application approval.
The sibling of operational ML in the “offline” world is analytical machine learning. These are applications that help business users make better decisions using machine learning. Analytic ML applications reside within an enterprise’s analytics stack and typically feed directly into reports, dashboards, and business intelligence tools.
Common examples include sales forecasting, churn forecasting, and customer segmentation.
Organizations use operational and analytical ML for different purposes, each with different technical requirements.
| analytical ML | Operational ML | |
|---|---|---|
| Automated decision making | human relations person | fully autonomous |
| decision-making speed | human speed | real time |
| optimization | large batch processing | Low latency and high availability |
| Main target audience | Internal business user | customer |
| power | Reports and dashboards | production application |
| example | sales forecast lead scoring customer segmentation Churn prediction |
Product recommendations Fraud detection Traffic forecast Real-time pricing |
Characteristics of analytical ML and operational ML
Operational machine learning practices
Let’s take a more concrete example of real-world operational machine learning at Uber Eats. When you open the app, you’ll see a list of recommended restaurants and suggest wait times for your order to arrive at your door. What looks simple inside the app is actually quite complex behind the scenes.
To ultimately display “Otto’s Tacos” and “20-30 minutes” in the app, Uber’s ML platform needs to look at a wide range of data from a variety of sources. Raw data source:
- How many drivers are currently near your restaurant? Are they delivering orders or handling on-going shipments?
- How busy is your restaurant’s kitchen right now? The more orders a restaurant is currently processing, the longer it will take to start processing new orders.
- Which restaurants have received high and low ratings from customers in the past?
- What cuisines, if any, are users currently actively searching for?
- And… where is the user’s current location?
michelangelo’s functional platform Convert this data to ML capabilities. these are, model has been trained on and used to create prediction In real time. for example, ‘num_orders_last_30_min’ It is used as an input feature to predict delivery times and is ultimately displayed in the mobile app.
The steps described above for converting raw data from a myriad of different data sources into features, and features into predictions, are common to all operational machine learning use cases. Whether the system is trying to detect credit card fraud, predict interest rates on car loans, suggest newspaper articles for the international affairs section, or recommend the best toys for 2-year-olds, the technical challenge is the same. And it is precisely this fundamental technological commonality that has allowed us to build. One central platform for all your operational ML use cases.
Trends enabling operational machine learning
Uber built its entire technology stack on modern data architecture and modern principles, so it was ready to take advantage of operational ML. Over the past few years, we’ve seen similar modernization occurring in far-flung corners of Silicon Valley.
Historical data is stored almost indefinitely
The cost of data storage has fallen significantly in recent years. As a result, businesses can now collect, purchase, and store information about every touchpoint with their customers. This is very important for ML. Training good models requires large amounts of historical data. Without data, there is no machine learning.
Data silos are being dismantled
From day one, Uber centralized nearly all of its data on a Hive-based distributed file system. Centralized data storage (or, alternatively, centralized access to distributed data stores) is important because it allows data scientists training ML models to know: what data is available, where to find it, and how To access. Most companies are not fully centralized all Their data (access) is not yet available. But architectural trends like The Modern Data Stack are bringing data scientists’ dreams of democratized data access closer to the spotlight.
Real-time data will be available in streaming
At Uber, we were fortunate to have Kafka, our “central nervous system” for data streams. Many real-time signals from services and mobile apps are streamed through Kafka. This is very important for operational ML.
You can’t detect fraud if you only know what happened yesterday. I need to know what happened in the last 30 seconds. Data warehouses and data lakes are built for long-term storage of historical data. And over the past few years, streaming infrastructures like Kafka and Kinesis have been adopted at scale to provide real-time signals to applications.
MLOps enables rapid iteration
At Uber, individual engineers are empowered to make daily changes to production systems. This process is supported by automation following DevOps principles. At Michelangelo, we introduced these principles to operational ML before the process was called MLOps 🙂. It was important to us that our data scientists could literally train a model within a day and safely deploy it to production.
Outside of Uber and far outside of Silicon Valley, a growing number of early adopters are bringing DevOps principles and automation to their software engineering as well as data science teams through MLOps. Of course, ML is still much more painful than software for most companies for the reasons outlined in this blog. However, I am confident that the industry is steadily moving toward a future where the typical data scientist at a typical Fortune 500 company can iterate on operational ML models multiple times a day.
The modern data architecture that enables operational ML looks like this:
If your organization has gone through some of the modernizations mentioned above (or if you’re starting one from scratch), you may be ready to start operational ML.
Get started with operational machine learning
In 2013, Uber wasn’t using machine learning in production. Tens of thousands of models are currently running in production. That change didn’t happen overnight.
If you want to leverage operational ML within your organization, we recommend the following steps:
Choose the right use case for machine learning
Not all problems can be solved with ML. Problem qualifiers that may be suitable for ML:
- Your system repeatedly makes many very similar decisions (at least tens of thousands)
- Making the right decision is not easy
- After a while after a decision has been made, it becomes possible to judge whether it is a good or bad decision.
When these factors are true, machine learning applications can make decisions, learn from those decisions, and continually improve.
Choose the use cases that actually matter
As mentioned earlier, the path to commercializing your first model is difficult. If you don’t see much future return on your first machine learning application, it’s easy to give up when the going gets tough. Priorities may shift, leadership may become impatient, and efforts may be short-lived. Choose the most likely use case.
Empower small teams and minimize stakeholders for your first model
The more handoffs involved in training and deploying a model, the more likely the project will fail. Ideally, you would start with a very small team of 2-3 people who have access to all the data they need, know how to train a simple model, and are familiar enough with the production stack to bring the application into production.
ML Engineers typically have a rare combination of data engineering, software engineering, and data science skills, making them ideal for paving the way. This is also a way to scale your machine learning team by incorporating a small group of ML experts into your product team.
