The same goes for machine learning.
Code, wait for results, interpret results, and go back to coding. Additionally, there will be some interim presentations about your progress. But just because things are pretty much the same doesn't mean there's nothing to learn. Quite the opposite! A few years ago, I started a daily habit of writing down the lessons I learned from my ML work. As I reflected on some of this month's lessons, I found three practical lessons that stood out to me.
- For new projects, choose your libraries wisely
- Save clips using Clipboard Manager
- read widely and read deeply
Choose between a library or your own code
Machine learning projects often start with the same question. Should I build everything myself or rely on existing libraries?
At the most basic level, this means deciding between PyTorch and a framework like TensorFlow. I was a big supporter of TensorFlow when Towards Data Science was still hosted on Medium. Currently I'm leaning more towards PyTorch. However, this section does not discuss such framework-level decisions.
Instead, I'd like to focus on project-level choices.
Imagine you are tasked with setting up a new ML project. The requirements are specific. sparsely labeled data, image input, and some architectural constraints. What should I do?
A good place to start is to search GitHub for projects that already meet most of your needs. If you find a 100% match, use it. If you can't find anything close, that's great too. Because the decision was clear that I needed to build it myself.
A more difficult case is when you find something but it's not a perfect fit. Do you want to patch your existing codebase until it works, or is it faster to implement your own solution from scratch?
There's no single right answer, but I've found a few rules of thumb to be helpful.
When fine-grained control is required Consider every aspect of your ML pipeline → Build it yourself.
If you just want a standard training pipeline →Use the library.
If you want to change an existing method → Start with a library that already exists.
When introducing your own method →Do it yourself.
Another factor worth considering is longevity. Code that you write yourself is code that you have complete control over. There are no sudden breaking changes or obscure bugs hidden in third-party repositories. Libraries, on the other hand, provide years of accumulated tests and optimizations that are difficult to reproduce on their own. The key is to balance current speed of progress with later maintainability.
In some cases, we've found that the best balance is to start with a library for rapid prototyping and then reimplement the important parts yourself once you know what works. That way, you can get early and rapid feedback, but maintain full ownership of the parts that matter most. In my experience, at least for research-intensive projects, the best libraries are something like: feel Something like a research code.
Two contrasting examples are the Avalanche library and the Mammoth library. Avalanche is more authentic and everything is nicely abstracted. Mammoth, on the other hand, is more like an extended research project, with direct control over the methodological part. Libraries like the latter give you the best of both worlds.
Although the above guidelines do not always solve the self-versus-library dilemma, they have enabled us to approach it more systematically. Over the years, and even this September, they have saved me from days of indecision.
Advantages of clipboard manager
Suppose you are controlling your ML project from the command line. Start execution as follows: python3 run.py --param1 --param2
Then create another one with different parameters. And one more thing. Now you want to juggle several runs and compare the results.
A simple method is to manually copy each output to a central location. Copy and paste. Copy and paste. At some point the wrong result will be overwritten and you will have to start over.
This exact situation happened to me earlier this month. When setting up a new project (after deciding whether to use a library or do it myself, see above), I also did some code tests. I wanted to check if everything runs without errors. So we evaluated several parameter settings, changing one or two arguments for each run. Since my project was an ML project and involved training an ML model, the script took a long time to run and I had to wait until I could test the next parameter. It was not possible to spin up individual runs because the cluster was occupied.
So in between testing two parameter settings, I focused on setting up the project and fixing bugs. Then, once I verified that the parameters were tested successfully, I scheduled the next parameter test and restarted the project setup.
As you can imagine, this strategy only works up to a point. After repeating this for a while, I no longer know which parameter combinations I've already tested. This was just a setup phase, so we hadn't implemented any actual testing and collecting results yet. This is usually done later. Fortunately, my habit of copying and pasting commands and changing arguments saved me from having to run my tests twice. Combine this with using a clipboard manager.
These tools not only save the most recent items, but also keep a history of everything you've copied. You can always go back and select the clips you want*.
Clipboard Manager's real strength lies in how it reduces cognitive overhead. You don't have to constantly worry, “Did I just overwrite the last copy?” Thinking, “Where did I save that snippet?” frees up your mental space for the actual task at hand. This is one of the little tools**. It doesn't look very big, but it will get stronger over time.
And importantly, this is not just an experiment. The same is true when preparing a talk, drafting a paper, or gathering numbers from multiple sources. If you use a clipboard manager long enough, you'll wonder how you ever worked without it.
I can prove it from my own experience. I've been using the Launchbar clipboard manager (but it's much more useful than this!) on my Mac machine for years. I installed the free Ditto utility on Windows. They often helped me when I clipped something and then deleted the original content (which I wanted to clip something to). The last clip was always available with one command, giving me the information I needed right away.
depth and breadth of reading
The same project also reminded me about reading papers. Setting this up required a combination of recent methodological advances and tabular data. As always, there was a ton of potentially relevant work. The question was what to read and what to skip.
This decision was easier than I expected. For the past few months, I have been reading papers regularly. I read it steadily, not intensively, but intermittently. This gave me a solid mental map of my field of study. More importantly, I was also reading adjacent research, papers not entirely from my area of expertise but addressing very similar challenges.
By reading more widely, I am able to identify connections between disciplines and recognize which methods are truly related. Without getting overwhelmed, I was able to quickly determine which papers were worth paying attention to and which papers I could safely ignore.
But the benefits go beyond efficiency (and knowledge, which is the main purpose of reading). Looking outside your main field often yields ideas you wouldn't have come across otherwise. For me, sometimes insights from adjacent fields end up forming the core of my project. In other words, breadth is not only a preparation for depth, but also a source of creativity.
Over time, the habit of reading from nearby fields builds resilience. Research fields change rapidly and methods jp fashion Today may be forgotten tomorrow. But if you're wide, you can adapt more easily. It already knows the adjacent fields and can move with the field instead of being carried away by it.
*Although not recommended, this is often the quickest method in the early stages of a project. At a later stage, it is recommended to record the results centrally.
** For Windows: Same as above. On Mac: Launch bar.
