Netflix has announced a new model that demonstrates how to use AI to remove objects from videos and edit the footage accordingly.
As outlined in a new paper, Netflix’s Video Object and Interaction Deletion (VOID) is a vision language model (VLM) designed to edit certain elements from a video and update the footage as if nothing had changed. As an example of this, Netflix showed an image of two cars racing towards each other. In the original video, the two cars collide, but with VOID, the car on the right side is removed, and the car on the left side is able to continue driving as if nothing happened.
Similar AI-powered tools exist such as Runway, Generative Omnimatte, and ProPainter, but Netflix is trying to show how VOID is better. The paper presents a picture comparing VOID with the aforementioned platforms. We also surveyed 25 people across five different scenarios and found that VOID was preferred 64.8 percent of the time, compared to Runway, which came in second at 18.4 percent. (The reason is unknown only 25 people were surveyed. )
Image credit: Netflix
Other examples include removing people from a video while preserving the shape of the object they are interacting with, such as a guitar or a ball. In many cases, objects can become deformed when human contact is removed, and Netflix is positioning VOID as a solution to that.
Of course, questions remain about this technology. First, the images shared by Netflix are from videos in apparently empty areas, such as the aforementioned car crash scene. It is unclear how well VOID would work if such a scene is set in a densely populated area such as a city.
And on a broader level, the question arises: Who would want such a feature? Producers will no doubt see this as a cost-cutting measure for reshoots and editing, but will filmmakers think so?
In any case, it is certain that technologies like VOID are not going away anytime soon. At Netflix in particular, the company recently spent a reported $600 million to acquire Interpositive, Ben Affleck’s AI startup that provides creators with unique tools to aid the filmmaking process. Affleck has previously spoken about the importance of using AI “responsibly” and putting filmmakers first, but it’s unclear how Netflix will use InterPositive going forward.
Source: Netflix Via: The Register
MobileSyrup may earn commissions from purchases made through our links, which help fund the journalism we provide for free on our website. These links do not influence our editorial content. Please support us here.
