Better than the 8 years I spent on machine learning. The more you learn about this field, the more you will be amazed at its diversity. When I started researching machine learning, all I knew about it was classical machine learning techniques like KNN and clustering algorithms. Now that I have more than 8 years of experience, I believe that the field is so vast that it is impossible for one person to understand everything.
However, I think there are some lessons that apply no matter where you are actually researching or practicing machine learning. I often take advantage of the opportunity to take another six months of progress in my machine learning journey and take a moment to pause and reflect on the past few years. What did I learn? Which lessons do you think will last?
This version of my Lessons learned article I looked back over the last eight years and tried to distill the lessons that keep coming up again and again. This time I realized that progress in machine learning, or in any field, often boils down to five things: they are patience, discipline, optimismgood projectand good team. These will be discussed in detail in the rest of this article.
patience
No one is born an expert in any field. Even if you weren’t born with genius-level abilities, and even if you were, being patient can work wonders. I often like to compare advances in machine learning to advances in sports activities. Beginners often fail very badly at first. The early days and months of a new activity.
For example, let’s say you want to learn to do a handstand. It takes a considerable amount of time to learn the balance of supporting weight over your head and to build up the muscle strength to begin with. And as you learn this, you will encounter many failures. You probably won’t see any progress in the early stages.
The same is true for machine learning. You may need to iterate the project several times until you get satisfactory results. Along the way, your work will be rejected and criticized until it’s deemed good enough. Of course this happened to me too. My paper was rejected four times in two years before finally being accepted at an A* conference.
During that time, I had to resubmit my paper multiple times and redo almost everything, including my story and experiments. I would be lying if I said I was always positive. Well, sometimes it might be better to throw away the paper and focus on something else. But patience. I focused on doing what I could at the time: redoing and resubmitting. And I’m waiting.
now It was accepted, but if I hadn’t been patient enough, it definitely wouldn’t have been published now.
optimism
Now move on to the next lesson. We need to cultivate a healthy dose of optimism (Or healthy ignorance). When working on a project such as a research paper or deployment, you will always face obstacles. It is necessary to continue, but simply continuing is not enough.
I think you need to be quietly optimistic about your work. Other people can give their opinion, and you can listen, but ultimately it’s your job. As long as you trust your progress, you’re usually fine. More precisely, as long as you trust the process most of the time, you’ll be fine.
discipline
Again, this leads nicely to the next lesson that has been the bedrock of my past few years: discipline. There are definitely more fascinating things in our daily lives. Well, sitting down and reading another paper is interesting, but checking Twitter or YouTube is more interesting. But that doesn’t move you forward, or even actively move you backwards.
To progress, you must ignore deviations and everyday distractions. You need a quality that has been admired in almost every branch of humanity: discipline. When studying for an exam, study regardless of the situation. When writing a lab report, write the report regardless of whether your coworker is at a party or not. When drafting your paper, do it regardless of whether other people are on vacation or not.
Almost finishing the report does not count. The same goes for most written papers. You can only move forward if you are focused and disciplined. My “secret” to this is to prioritize what’s important every day. We try to stick to this schedule 80% of the time to make it realistic and doable. I found it to work pretty well.
project
This is a lesson specific to those working toward a (doctoral) thesis. You need a good enough project to work on. If you’re intelligent enough, it might not matter too much. More importantly, your topic is niche enough, new enough, and stable enough that other people are interested in it and can leverage their strengths in it.
A few years ago, I had a colleague who was researching emergent features in LLM. This is a very interesting topic and I learned about it after seeing some posters at ICLR 2024 in Vienna. My colleague had very good knowledge of transformer models and programming skills. But the field was too new and too unstable. The model progressed too quickly for him to grasp the topic. Eventually, it burned him out and he switched to other things. (He’s fine now!)
team
I saved the most important lesson for last. Who you work with is often more important than what you work with. (If you tick both boxes, that is, you have a good group and your work was great, you are especially lucky). If you’re great at what you do, but the environment doesn’t allow you to take advantage of it, you’re in the wrong place. If you can’t make mistakes, you’re in the wrong place. If you’re afraid to show up, you’re in the wrong place.
There is a term for team effectiveness. You (and your team) need to be in your comfort zone first. This means being energetic and working well together. Only after that can you move on to the high-power zone, where you can produce good work under pressure. Make sure you are there too.
lastly
If you look at the lessons, there is nothing specific to machine learning. Rather, they are the same old ones that have defined good human progress over the past millennia. This is not a bad thing. In fact, the opposite is true. If technology were to completely overhaul everything that has accompanied us since then, we would be lost. So it’s a relief to see the classic lesson recreated once again.
- be patient
- be optimistic
- correct discipline
More passionate lessons
- Engage in work that utilizes your strengths
- Choose a good and healthy team
Nothing new, everything related.
