Machine learning and AI are one of the most popular topics these days, especially within the technical field. I am fortunate to work and develop with these technologies every day as a machine learning engineer!
In this article, I will throw away the light and advice on how you will become yourself as I walk my journey to becoming a machine learning engineer!
My background
In one of my previous articles, I wrote extensively about my journey to secure my first data science job from school. I recommend checking out that article, but here's a summary of the important timeline.
Most people in my family studied some sort of STEM subject. My great grandchild is an engineer, my grandparents study physics, and my mother is a mathematics teacher.
So my roads were always paved for me.
After seeing Big Bang theory at age 12, I decided to study physics in university. It's fair to say that everyone was so proud!
At school, I was never stupid. I was actually relatively bright, but didn't apply myself completely. I got decent grades, but definitely not what I was completely capable of.
I was very arrogant and thought I would do well at work.
I applied to top universities such as Oxford and Imperial College, but given my work ethic, I was paranoid, thinking I had a chance. On the day of the result I missed my offer and I was cleared. This was probably one of the saddest days of my life.
Liquidation in the UK is where universities provide locations for students in specific courses with space. It is primarily intended for students who do not have university offers.
I was fortunate to be offered the opportunity to study physics at the University of Surrey and continued to earn my first-class master's degree in physics!
There is no alternative to diligence. That's a nasty cliché, but it's true!
My original plan was to get a doctorate and become a full-time researcher or professor, but during my degree I was in research years and I just felt that a research career wasn't that for me. Everything moved very slowly, and it didn't seem like there was much opportunity in the space.
During this time, DeepMind released them Alphago – Movie The YouTube documentary appeared on my home feed.
From the video I began to understand how AI worked and began to learn about neural networks, reinforcement learning, and deep learning. To be honest, to this day I am not an expert in these fields yet.
Naturally, I dig deeper and found that data scientists use AI and machine learning algorithms to solve problems. I was hoping for it and started applying for the Data Science Alumni role.
I spent countless hours taking courses and working on projects. I applied Over 300 jobs And eventually landed my first Data Science Graduate Studies scheme in September 2021.
You can hear more about my journey from the podcast.
A journey of data science
I started my career at an insurance company. There, we built a variety of monitored learning models, mainly using gradient boost tree packages such as CatBoost, Xgboost, and Generalized Linear Models (GLMS).
We have built a model to predict:
- scam -Has someone fraudulently claimed profit?
- Risk price -What premium should you give to someone?
- Number of claims-How many claims will someone make?
- Average Cost of Billing -What is the average bill value someone has?
We created approximately six models that span regression and classification spaces. I learned a lot here, especially with statistics. I worked very closely with the actuaries, so my mathematics knowledge was excellent.
However, due to the company's structure and setup, it was difficult for my model to advance past the POC stage. So I felt that I understood the “technical” side of the toolkit and how companies use machine learning in production.
A year later, my previous employer asked me if I wanted to apply for the role of a junior data scientist, specializing in time series prediction and optimization issues. I really liked the company and after some interviews I was offered a job!
I worked for this company for about 2.5 years. There, he became an expert in forecasting and combination optimization problems.
I developed many algorithms and deployed models through AWS using best practices in software engineering, such as unit testing, low environments, shadow systems, and CI/CD pipelines.
It's fair to say I've learned a lot.
I have worked very closely with software engineers, so I took up a lot of engineering knowledge and continued with spontaneous machine learning and statistics.
I got a promotion from junior to mid-level at that point!
Migration to MLE
Over time I realized that the real value of data science is to use it to make live decisions. There's a good quote by Pau Labarta Bajo
The business value of the ML model in a Jupyter notebook is $0
If there are no results, there's no point in building a truly complex and sophisticated model. In many cases, adding 0.1% accuracy by staking multiple models is often not worth it.
Sometimes it's better to build something simple that can be expanded.
With this in mind, I began to think about the future of data science. My mind has two ways.
- analysis-> You are primarily working to gain insight into what your business should do and what you should look for to improve its performance.
- engineering-> Ships solutions that bring business value (models, decision-making algorithms, etc.).
Data scientists who analyze and build POC models feel that they will become extinct in the coming years as they do not provide tangible value to their businesses, as mentioned above.
That's not to say they're completely useless. You have to think about it from a business perspective of their return on investment. Ideally, the value you bring should be more than your salary.
You would like to say you did an “x that produced x” that can be done using the two methods mentioned above.
The engineering side was the most interesting and enjoyable thing for me. I really benefit people and I really enjoy coding and building what they can use.
To move to the ML engineering side, I asked the line manager if I could deploy the algorithms and ML models that I was building myself. I get help from software engineers, but I write all the production code, design my own system, and set up the deployment process independently.
And that's exactly what I did.
I have basically become a machine learning engineer. I developed the algorithms and then shipped them to production.
I also adopted NeetCode's data structure and algorithm course to improve the fundamentals of Computer Science and I began blogging about the concepts of software engineering.
Coincidentally, my current employer contacted me around this time and asked if I wanted to apply for the role of a machine learning engineer specializing in ML and optimization in their company!
I call it luck, but obviously the universe was saying something to me. After a few rounds of interviews, I was offered the role and I am now a full-fledged machine learning engineer!
Luckily, I created my luck by documenting my upskilling and learning, despite the role that “fallen on me.” That's why I always tell people to show their work. You don't know what's coming from that.
My advice
I would like to share some of the main advice that will help you move from machine learning engineer to data scientist.
- experience– Machine learning engineers are do not have In my opinion, entry level position. You need to be familiar with data science, machine learning, software engineering and more. You don't have to be an expert on everything, but you have a great basics across the board. Therefore, it is recommended that you have several years of experience as a software engineer or data scientist, and as a other field of self-study.
- Production code– If you are from data science, you need to learn to write appropriate, well-tested production code. You need to know typing, lint, unit testing, formatting, mocking, CI/CD and more. It's not that difficult, but it requires some practice. I would recommend asking your current company to work with a software engineer to gain this knowledge, it worked for me!
- Cloud System– Most companies now deploy many architectures and systems in the cloud, and machine learning models are no exception. Therefore, it is best to practice using these tools and understand how your model can make it live. Honestly, I've learned most of this at work, but there are courses you can take.
- Command Line– I'm sure most people already know this, but all technical experts need to become proficient in the command line. Use extensively when developing and writing production code. Here is a basic guide you can check out.
- Data Structures and Algorithms– Understanding the basic algorithms of computer science is very useful in the role of MLE. Mostly because you are likely to be asked about it in an interview. It's not that difficult to learn compared to machine learning. It will take time. Every course will do tricks.
- git & github– Again, most tech experts need to know GIT, but it's essential as an MLE. A way to crush commits, do code reviews, and write unresolved pull requests is essential.
- Specialize– Many of the MLE roles I saw had to specialize in a particular field. I specialize in time series prediction, optimization, and general ML based on my previous experience. This helps you stand out in the market, and most companies are looking for experts these days.
The main theme here is that I have essentially matured my software engineering skills. This makes sense as I already have all the mathematics, statistics and machine learning knowledge from being a data scientist.
If I'm a software engineer, the migration could be the opposite. This is why securing the role of a machine learning engineer can be extremely difficult as it requires proficiency across a wide range of skills.
Summary and more thoughts
I have a free newsletter, Data PlateI share weekly tips and advice as a practice data scientist. Plus, subscribe and you'll get me Free Data Science Resumeand Short PDF version of my AI roadmap!
