Meta AI Open Source DINOv2: A New AI Method for Training High-Performance Computer Vision Models Based on Self-Supervised Learning

With recent AI developments, basic computer vision models are sometimes pre-trained using large datasets. Creating general-purpose visual functions, or functions that work across image distributions and jobs without tweaks, could greatly simplify the use of images in any system, and these models stand out in this regard. is quite promising. This study shows that such features can be produced by current pre-training approaches, especially self-supervised methods, when trained on sufficient curated data from a variety of sources. . Meta AI announced his DINOv2. This is the first self-supervised learning method for training computer vision models that achieves performance equal to or better than the gold standard.

These visual properties are stable and work well across domains without tweaking. They are generated using DINOv2 models and can be used directly in classifiers as basic as linear layers in various computer vision applications. The pre-trained model was fed 142 million photos without labels or comments.

Self-supervised learning, the same approach used to develop state-of-the-art big language models for text applications, is a promising candidate for training AI models because it does not require huge amounts of labeled data. A powerful and versatile method. Models trained with the DINOv2 process are similar to previous self-supervised systems as they do not require information related to the photos in the training set. Imagine being able to learn from any given image, not just images with a given set of tags, a given set of alt text, or a given caption.

essential features

• DINOv2 is a new approach to building high-performance computer vision models using self-supervised learning.
• DINOv2 provides high-quality unsupervised learning of visual features that can be used for both image-level and pixel-level visual tasks. Image classification, instance finding, video understanding, depth estimation, and many other tasks are covered.
• Self-supervised learning is the main attraction here, as DINOv2 allows us to build a generic and flexible framework for a wide variety of computer vision tasks and applications. There is no need to fine-tune the model before applying it to different domains. This is the pinnacle of unsupervised learning.
• Creating large, highly curated and diverse datasets for training models is also an integral part of this research. There are 142 million photos in the data collection.
• A more efficient implementation that reduces factors such as memory utilization and processor requirements is another algorithmic effort to stabilize training of larger models.
• The researchers have also published a pre-trained model for DINOv2. The ViT model checkpoints published on PyTorch Hub are also included in the Vision Transformer model pre-training code and recipes.

advantage

• A simple linear classifier can take advantage of the high-performance features provided by DINOv2.
• You can take advantage of the adaptability of DINOv2 to build a versatile infrastructure for a variety of computer vision applications.
• The features perform significantly better than state-of-the-art depth estimation methods in- and out-of-domain.
• Skeletons remain generic without tweaks and the same functionality can be used in many activities simultaneously.
• The DINOv2 model family performs on par with weakly supervised features (WSL). It is a significant improvement over the traditional state-of-the-art in self-supervised learning (SSL).
• The features generated by the DINOv2 model are useful out of the box, demonstrating the model’s excellent out-of-variance performance.
• DINOv2’s reliance on self-monitoring means it can interrogate any image database. In addition, it can pick up aspects that the status quo method can’t, such as depth estimation.

Having to rely on human annotation of the photos is a hindrance as it reduces the data available for model training. Images can be very difficult to classify in highly specialized application areas. For example, it is difficult to train machine learning models using labeled cell imaging, as more experts are required to annotate cells at the required scale. To facilitate comparisons between established and emerging therapies, for example, self-administered training in microscopic cytographs paves the way for basic cell imaging models and, in turn, biological discovery.

To build a large pre-training dataset from such sources, it is important to discard irrelevant photos and balance the dataset between concepts. Training more complex architectures is an important part of this effort, and these models need access to more information in order to improve their performance. However, details are only available in some cases. Researchers conducted their research using publicly available collections of crawled web data. They created a meaningful data selection process inspired by LASER, as no curated data set was large enough to meet their demands.

The next step is to use this model as a building block for more sophisticated AI systems that can interact with real language models. If a complex AI system has access to a visual backbone that provides a wealth of information about an image, it can make more thorough inferences about the image than a single text phrase could.

check out paper, Demo, Github, and Reference articledon’t forget to join Our 19k+ ML SubReddit, cacophony channeland email newsletterWe share the latest AI research news, cool AI projects, and more. If you have any questions about the article above or missed something, feel free to email me. Asif@marktechpost.com

🚀 Check out 100 AI Tools in the AI ​​Tools Club