How to perform effective data cleaning for machine learning

The most important steps you can perform in your machine learning pipeline. Without data, improvements to the model algorithm are not important. After all, the word “trash, garbage output” is not just a proverb, it is a true inherent in machine learning. Without the right high quality data, it will be difficult to create high-quality machine learning models.

This article explains how to effectively apply data cleaning to your own datasets to improve the quality of fine-tuned machine learning models. We will explain why data cleaning and data cleaning techniques are necessary. Finally, we also provide important notes to keep in mind, such as maintaining a short experimental loop.

You can also read the article Openai Whisper for transcriptionjoin Nvidia GTC Paris 2025 and create powerful embeddings for machine learning.

table of contents

motivation

My motivation for this article is that data is one of the most important aspects of working as a data scientist or ML engineer. This is why companies like Tesla, Deepmind and Openai are focusing on annotating data. For example, Tesla had around 1,500 employees working on data annotation for fully automated driving.

However, if you have a low-quality dataset, you will have a hard time using a high-performance model. This is why cleaning data after annotation is so important. Cleaning is essentially the foundational block of all machine learning pipelines, including training the model.

meaning

To be explicit, data cleaning is defined as a step that is performed after the data annotation process. So we already have a series of samples and corresponding labels, and we aim to clean these labels to ensure accuracy.

Furthermore, words Note and labeling It is often used interchangeably. I think they mean the same thing, but for consistency, I use only annotations. Using data annotations means the process of setting labels on data samples. For example, if you have an image of a cat, annotating the image means setting annotation cat Images are supported.

Data cleaning techniques

It is important to note that if the dataset is small, you can choose to have a second look at all samples and annotations. However, in many scenarios, data annotations take too long, so this is not an option. This is why we list some techniques below to perform data cleaning more effectively.

Clustering

Clustering is a common, unsupervised technology in machine learning. Clustering allows you to assign a set of labels to data samples without having the original dataset for samples or annotations.

However, clustering is also a great data cleaning technique. This is the process used to perform data cleaning in clustering.

1. Embed all data samples. This can be done using a combination embedding such as text embedding using the BERT model, visual embedding using a squeeze net, or clip embedding in OpenAI. The point is that a numerical representation of the data sample is required to perform clustering
2. Apply clustering techniques. Also, unlike db scans, I like K-means because I assign clusters to all data samples. (Outliers can fit many scenarios, but are best for data cleaning). If you are using K-means, try different values for parameter k.
3. You will now see a list of data samples and assigned clusters. Next, iterate through each cluster to see if the labels are different within each cluster.

I'll explain in detail in step 3. Please use the example. Use a simple binary classification task to assign images to labels

Now we have 10 images with the following cluster allocations: A small example is seven data samples with two cluster allocations. In the table, the data sample looks like this

If visualization is possible as follows:

Next, we use a for loop to go through each cluster and decide which sample we want to explore further (see Python code for this)

• Cluster A: In this cluster, all data samples have the same annotation (CAT). Therefore, the annotations are more likely to be correct. I Please don't A secondary review of these samples is required
• Cluster b: We definitely want to look into this cluster sample in more detail. Here is an image with embeddings placed closely in the embedded space. This is highly doubtful as similar embeddings are expected to have the same label. Look at these four samples frequently

Can you see how I had to use the data sample from 4/7?

This is a way to save you time. Only the data samples that are most likely to be incorrect can be found. This technique can be scaled to thousands of samples with more clusters, saving enormous amounts of time.

---

I'll also provide the code for this example to highlight how to do clustering in Python.

First, let's define the mock data.

sample_data = [
    {
        "image-idx": 0,
        "cluster": "A",
        "label": "Cat"
    },
    {
        "image-idx": 1,
        "cluster": "A",
        "label": "Cat"
    },
    {
        "image-idx": 2,
        "cluster": "A",
        "label": "Cat"
    },
    {
        "image-idx": 3,
        "cluster": "B",
        "label": "Cat"
    },
    {
        "image-idx": 4,
        "cluster": "B",
        "label": "Cat"
    },
    {
        "image-idx": 5,
        "cluster": "B",
        "label": "Dog"
    },
    {
        "image-idx": 6,
        "cluster": "B",
        "label": "Dog"
    },
    
]

Next, repeat all clusters to find the samples you need to consider.

from collections import Counter
# first retrieve all unique clusters
unique_clusters = list(set(item["cluster"] for item in sample_data))

images_to_look_at = []
# iterate over all clusters
for cluster in unique_clusters:
    # fetch all items in the cluster
    cluster_items = [item for item in sample_data if item["cluster"] == cluster]

    # check how many of each label in this cluster
    label_counts = Counter(item["label"] for item in cluster_items)
    if len(label_counts) > 1:
        print(f"Cluster {cluster} has multiple labels: {label_counts}. ")
        images_to_look_at.append(cluster_items)
    else:
        print(f"Cluster {cluster} has a single label: {label_counts}")

print(images_to_look_at)

This makes it just a review now Images_to_look With variables

CleanLab

CleanLab is another effective technique that can be applied to clean your data. CleanLab is a company that provides products that detect errors in machine learning applications. But they are also discussing here by open source GitHub tools and running data to clean themselves.

Essentially, CleanLab retrieves the data and analyzes the input embedding (for example, created with Bert, Squeezenet, or Clip) and output logit from the model. Next, perform a statistical analysis of the data to detect the samples that are most likely to have the wrong label.

CleanLab is a simple tool to set up as it only provides input and output data and requires handling of complex statistical analysis. I've seen how CleanLab has the powerful ability to detect samples with potential annotation errors.

Given that they have the ability to utilize excellent README, I leave the implementation of CleanLab to the reader.

Prediction and comparison with annotations

The last data cleaning technique I experience is to predict the sample using a finely tuned machine learning model and compare it with annotations. Essentially, you can use techniques such as K-fold Cross-validation. Here, you can split the dataset into several times of different trains and test splits, and predict the entire dataset without leaking the test data into the training set.

After predicting the data, you can compare the predictions with the annotations you have in each sample. If the prediction corresponds to annotations, you do not need to review the sample (it is less likely that this sample will have an incorrect annotation).

Technique overview

Here we have introduced three different techniques

• Clustering
• CleanLab
• Prediction and comparison

The main point of each of these techniques is to exclude samples that are likely to be incorrect and only check those samples. This saves a lot of time spent reviewing your data simply by looking at a subset of data samples. Different techniques fit better in different scenarios.

Of course, you can combine either technique with Union or intersection:

• Use Union To find samples that are likely to be incorrect between samples found using different techniques
• I believe that using intersections between samples is wrong to check for samples that I think are wrong

It is important to keep in mind

You also need a short section of important points to keep in mind when performing data cleaning

• Quality>Quantity
• A short experimental loop
• The effort required to improve accuracy increases exponentially

I'll now explain each point in detail.

Quality>Quantity

When it comes to data, having a data set of is much more important Correctly annotated Some incorrectly annotated samples, not larger datasets. The reason is that when you train a model, it blindly trusts the annotations you assigned and adapts the weights of the model to this ground truth

For example, imagine there are 10 images of dogs and cats. The nine images are correctly annotated. However, one of the samples shows an image of a dog, but in reality it is a cat. You are now telling the model that when it sees a dog, it should update its weight as it predicts cat Instead. This naturally reduces the performance of the model significantly and needs to be avoided at any cost.

A short experimental loop

When working on a machine learning project, it is important to have a short experimental loop. This is because in many cases you will need to try different configurations of hyperparameters or other similar settings.

For example, if you apply the third technique described above using a model and compare the output with your own annotations, it is recommended to frequently retrain the model with cleaned data. This will improve model performance and further improve incorrect annotations.

The effort required to improve accuracy increases exponentially

It is important to note that if you are working on a machine learning project, you need to be aware of what your requirements are in advance. Do you need a model with 99% accuracy, or is 90% sufficient? If 90% is enough, you could save a lot of yourself, as you can see in the graph below.

The graph is a sample graph I created and does not use actual data. However, it highlights some important notes I made while working on the machine learning model. In many cases, you can reach 90% accuracy (or what you define as a relatively good model). Of course, accurate accuracy depends on the project. However, pushing that accuracy to 95% or 99% requires exponentially more work.

For example, if you start cleaning your data first, you will see rapid improvements when you retrain and retest the model. However, as you do more and more data cleaning, your returns will likely decrease. Keep this in mind when working on a project. Also, prioritize where you spend your time.

Conclusion

In this article, we discussed the importance of data annotation and data cleaning. We have introduced three methods to apply effective data cleaning.

1. Clustering
2. CleanLab
3. Prediction and comparison

Each of these techniques helps to detect data samples that are likely to be incorrectly annotated. Depending on the dataset, the effects of different techniques vary, and you usually need to try them out to see what's best for you and what's working on.

Additionally, we discussed important notes to keep in mind when performing data cleaning. Remember that having a high-quality annotation is more important than increasing the amount of annotations. With that in mind, ensure a short experimental loop where you can clean some data, retrain the model, and test it again. Machine learning models can improve performance rapidly.

Follow me about social society:

🧑‍💻Please contact us
🌐 Personal blog
🔗LinkedIn
🐦X / Twitter
medium✍
🧵Thread