Meta, the parent company of Facebook, has announced Segment Anything Model 2 (SAM 2), an AI model that can identify and track objects in images and videos. This latest model builds on the original SAM model that focused solely on image segmentation.
“SAM 2 can segment any object and track it consistently in real-time across all frames of video, opening up new possibilities in video editing and enabling new experiences in mixed reality,” the company said in a blog post. SAM 2 can segment any object in an image or video and track it consistently in real-time across all frames of video.
What is segmentation and how it helps in video and photo editing
Segmentation can be described as identifying image pixels that belong to an object. For example, in a video of a soccer player dribbling, the player's cleats can be segmented. The color or style of those cleats can then be edited. According to Meta, segmentation is used by researchers in tasks such as scientific image analysis and photo editing.
“Our original Segment Anything model, which we released last year, served as the inspiration for new AI-enabled image editing tools in our apps, including Backdrop and Cutouts on Instagram. SAM has powered diverse applications in science, medicine and many other industries,” Mehta said.
According to the company, SAM is used in marine science to segment sonar images to analyze coral reefs, in disaster relief to analyze satellite images, and in medicine to segment cellular images to detect skin cancer.
The technology has potential applications in a variety of fields, including video editing, mixed reality, scientific research and autonomous vehicles.
Meta is publishing its research results “so that others can explore new features and use cases.”
“SAM 2 can segment any object and track it consistently in real-time across all frames of video, opening up new possibilities in video editing and enabling new experiences in mixed reality,” the company said in a blog post. SAM 2 can segment any object in an image or video and track it consistently in real-time across all frames of video.
What is segmentation and how it helps in video and photo editing
Segmentation can be described as identifying image pixels that belong to an object. For example, in a video of a soccer player dribbling, the player's cleats can be segmented. The color or style of those cleats can then be edited. According to Meta, segmentation is used by researchers in tasks such as scientific image analysis and photo editing.
“Our original Segment Anything model, which we released last year, served as the inspiration for new AI-enabled image editing tools in our apps, including Backdrop and Cutouts on Instagram. SAM has powered diverse applications in science, medicine and many other industries,” Mehta said.
Expanding
According to the company, SAM is used in marine science to segment sonar images to analyze coral reefs, in disaster relief to analyze satellite images, and in medicine to segment cellular images to detect skin cancer.
The technology has potential applications in a variety of fields, including video editing, mixed reality, scientific research and autonomous vehicles.
Meta is publishing its research results “so that others can explore new features and use cases.”
