Segmentation (identifying which image pixels belong to an object) is useful for tasks such as scientific image analysis and photo editing. Segmentation Model The release of AI-enabled image editing tools last year brought new AI-enabled image editing tools to our apps. background and Crop On Instagram, SAM has facilitated diverse applications in science, medicine, and many other industries. For example, SAM is used for segmentation in marine science. Sonar Image and Analyzing Coral Reefs, Satellite image analysis In the disaster relief and medical fields, Cell Images And help Skin Cancer Detection.
Our new Segment Anything Model 2 (SAM 2) extends these capabilities to video. SAM 2 can segment any object in an image or video and track it consistently in real-time across all frames of the video. Existing models have not been sufficient to achieve this because segmentation in video is much harder than in images. In video, objects can move fast, change appearance, or be obscured by other objects or parts of the scene. In building SAM 2, we have solved many of these challenges.
“We believe this research opens up new possibilities, including easier video editing and creation, to create new experiences in mixed reality. SAM 2 can also be used to track target objects in video to speed up the annotation of visual data for training computer vision systems, including those used in autonomous cars. It can also enable creative ways to select and manipulate objects in real-time or live video.”
Adhering to an open science approach, we are sharing our research findings on SAM 2 to allow others to explore new capabilities and use cases, and we look forward to seeing how the AI community uses this research.
