What are we doing here?
Now that the winter break is over and many of us are returning to the office and getting back to work, I've been thinking a bit about the relationship of machine learning functions to the rest of the business. As I've been settling into my new role at DataGrail since November, I've been reminded of how important it is for a machine learning role to know what the business actually does and what it needs.
My thoughts here aren't necessarily applicable to all ML practitioners — those who are purely academics can probably move on — but for those whose role is to leverage ML for business or organizational purposes, rather than for the advancement of ML itself, I think it's worth thinking about how you engage with the organization you work for.
The question I want to ask is why did someone decide to hire your skill set here, why did you need new personnel? New hires aren't cheap, especially in a technical role like ours. Even if you were replacing someone who left, that's not guaranteed to happen these days, and maybe you had a specific need. How did you explain to your finance guy that you needed to hire someone with machine learning skills?
You can learn a few useful things by exploring this question. First, what is the ideal outcome people expect from you? They want to be more productive with data science or machine learning, but if they don't know what that is, it can be hard to meet those expectations. You can also learn something about your company culture from this question. Once you know how they think about the value of hiring new ML talent, is that thinking realistic about the contribution ML can make?
Beyond meeting these expectations, you need to have a unique view of what machine learning can achieve within your organization. This requires researching your business and talking to many people across different functional areas (in fact, this is where I spend a lot of my time right now, as I am answering this question in my role). What are you trying to do with your business? What is the formula you believe will lead to success? Who is your customer and what is your product?
Somewhat related to this, you also need to look at data: what data does the business have, where is it, how is it managed, etc. This will be crucial to assess exactly what efforts should be focused on in this organization. We all know that having data is a prerequisite to do data science, but if the data is disorganized or doesn't exist at all, you need to take that into account and talk to stakeholders about reasonable expectations for machine learning goals. This is part of bridging the gap between the business vision and the machine learning reality, and is often overlooked when everyone wants to hit the ground running developing new projects.
Once you know these answers, you need to bring your perspective to the table on how elements of data science can help. Don’t assume that everyone already knows what machine learning can do. They probably don’t. Other roles have their own areas of expertise and it’s unfair to assume that they also know the intricacies of machine learning. This is a really fun part of the job because it lets you explore creative possibilities. Is there a hint of a classification problem somewhere, or a prediction task that could really help one department succeed? Is there a ton of data somewhere that could possibly have useful insights, but that nobody has had time to dig into? Maybe an NLP project is lurking amongst a bunch of unorganized documents.
Understanding your business goals and how people want to achieve them will help you connect machine learning to those goals. You don't need a one-size-fits-all solution that will solve all your problems overnight, but if you can draw the line between what you want to do and what everyone wants to achieve, you'll have much more success integrating your work with the rest of the company.
This may seem like a bizarre question, but in my experience, it's extremely important.
If your work is not aligned with the business and understood by your colleagues, it will be misused or ignored, and the value you could have contributed will be lost. If you read my column regularly, you know that I am a big advocate of data science literacy and believe that DS/ML practitioners have a responsibility to improve it. Part of your job is to help people understand what you have created and how it helps them. It is not the responsibility of finance or sales to understand machine learning without education (or “enablement” as it is often said these days). It is your responsibility to bring the education.
If you're part of a relatively mature internal ML organization, this might be easier; hopefully, this literacy has been picked up by others before you. But it's not a guarantee. Even large, expensive ML capabilities within an enterprise can be siloed, isolated, and indecipherable to other parts of the company. This is a terrible situation to be in.
What should you do about this? There are a few options, and it depends a lot on the culture of your organization. Take every opportunity to talk about your work, and try to speak at a level that a layperson can understand. Explain definitions of technical terms, not just once, but many times, because these things are hard and people need time to learn. Create documentation in whatever wiki or documentation system your company uses so people can refer to it when they forget. Offer to answer questions, and answer honestly, openly, and friendly, even if the question seems too simple or misguided. Everyone has to start somewhere. If there is a basic level of interest from your colleagues, you can set up learning opportunities such as Lunch and Learns and discussion groups on broader ML-related topics, not just the specific project you're currently on.
Moreover, it's not enough to explain all the great things about machine learning — you also need to explain why your colleagues should care about it and how it relates to the overall business and their personal success. What does machine learning bring to your colleagues that will make their jobs easier? You need to have a good answer to that question.
I've discussed this from the perspective of how to get started in a new organization, but even if you've been working with machine learning in your business for a while, it's useful to revisit these topics and take stock. Making the role effective isn't a one-and-done thing, it requires ongoing attention and maintenance. But with persistence, it gets easier, as your colleagues learn that machine learning isn't scary, but can help with their work and goals, and that your department is helpful and collaborative, rather than unclear and siloed.
To summarise:
- Find out why your company adopted machine learning and explore the expectations behind that choice.
- Understanding what the business does and its goals is essential to doing work that contributes to the business (and keeps yourself relevant).
- People aren't going to magically understand it automatically, so you have to help them understand what you're doing and how it will benefit them.
