As an AI team manager, Vivek Gupta stays well-informed to effectively guide AI professionals and drive his team forward. In his talk “Growing and developing strong machine learning engineers” at Dev Summit in Boston, Mr. Gupta said that engineers need feedback on both technical and interpersonal skills. He values learning time, asking for help, and collaboration between teams. Mentorship, data processing, and human validation are the keys to success for machine learning engineers.
As the manager of an AI team, he needs to know a little bit of everything, said Gupta. At least to understand where the value is, you need to be well-versed in applied science, and you need to stay up to date with the latest information. He added that his senior engineers are experts at digging deep, but he also needs to have ideas to move the team forward.
One of the main things engineers are looking for is feedback. They are fresh out of school, used to getting good grades, and want to know how they can do better, Gupta explained.
The feedback is very diverse. Some of it is about how you approach the coding field, and some of it is about how you actually interact with other people or how you deal with other people and teams that you work with.
To develop engineers, Gupta said, they should be given time to learn, try and practice new things.
Mr. Gupta said we need to get engineers to ask questions. Usually they don't ask questions until they've been stuck for a really long time, he added. You should actually encourage them to go to senior engineers and managers and ask if they know someone who can unblock them.
To foster collaboration, Gupta explained that he wants people to talk to people from other disciplines and other projects.
Other teams often have ideas that can leverage work done by someone else or share work they've done to reduce duplicative work. Encourage this type of collaboration and listen to others' stories and project design presentations so you can learn from them.
Senior engineers can become mentors to juniors. Mr. Gupta said that coaching seniors on how to do mentorship can make an organization more scalable.
People working on machine learning in production need to understand how data scientists do AI and machine learning. Gupta said you also need to know about data management for machine learning, which is different from machine learning.
You need to keep track of the data used to train the model and the test sets that may be used to validate the model. Data needs to be moved from one place to another, reformatted, and aggregated.
Consistency in how you manage data for training is important. Gupta suggested automating frequent retraining by building a training pipeline.
Human involvement is required to verify answers, check generated code, and compare different alternatives. Mr. Gupta said user feedback is what closes the loop. He concluded that “thumbs up” and “thumbs down” don't just tell you how well you're doing your job; they provide feedback about how your model is performing and which models need modification.
InfoQ interviewed Vivek Gupta about the rise of machine learning engineers.
InfoQ: What do you do to help your engineers learn and try new things?
Vivek Gupta: We regularly host hackathons with our team and participate in Microsoft-wide hackathons every year. Additionally, we have a day of learning time at the end of each sprint (in our case, a two-week sprint). Our team may also hold lunch and learn sessions to share what we've learned and invite guest speakers. Much of the recent learning has been around agents and using AI assistance for coding. In these areas, each has experience, so each has the opportunity to demonstrate new things he or she has learned.
While much of our learning focuses on technology, there are other aspects to their learning and development. We offer them the opportunity to learn about managing their careers, the work of a manager/technical lead, and how to measure their impact. This often takes the form of inviting hands-on opportunities with more senior speakers, previous cohort members, interns, or even supporting another team as a PR reviewer or technical advisor.
InfoQ: What does collaboration look like for senior engineers on your team?
gupta: For senior engineers, collaboration means knowing what's going on across the team, helping with PR reviews, participating in design reviews for each project, and helping lead learning sessions for new team members. This fosters knowledge sharing and develops people across teams as natural technical leaders. Junior team members often find it easier to go to a senior engineer than to go to a manager.
InfoQ: How can MLOps help you properly manage large language models?
gupta: Large-scale language models suffer from some of the same problems as traditional models. As we are currently fine-tuning them, we need to track the data we used to fine-tune them. You need a pipeline for evaluating prompts, and you need to maintain a library of prompts for different models. Although LLM works differently, the MLOps learnings continue to be applied to ensure that our approach is a well-designed approach for production scenarios using LLM.
