The phrase “fix with post!” has long been curses, memes, horror lines, and what most filmmakers avoid saying at all costs. Still, the editors find themselves working on issues that need quick fixes. Random logos, unnecessary elements of shots, faces of people who didn't sign the release… especially in smaller productions like corporate movies and YouTube videos, the editors are responsible for. In the past, some of these footage problems could not be solved without elaborate VFX skills. Recently, there is AI. Or, to be precise, AI masking.
Of course, it is not a substitute for proper filming planning or professional VFX compositor expertise. However, in everyday situations where you need a little repair or Polish to your shot, AI video masking is one of the handy tools you can use at your disposal. Let's take a look at some examples of where and how it can be useful.
To that end, I'm using the recent Mzed course, “The Efficive Filmmaker: AI Video Masking for us for the of the Lostons lessons. So I'll explore these tools in detail and include a thorough, step-by-step guide. So if you're not familiar with the topic at all, check it out!
AI Masking Applications: A General Approach
Using AI to select something in a shot is not a one-button task. However, it is easier than rotoscope each frame of an object. So let's give it a try! For simplicity, this article uses Davinci Resolve's Magic Mask. If you're editing in a different NLE, don't worry. It is likely that there are similar tools with similar processes. (For example, we will also explain magnetic masks and rot brushes on the course).
Before applying an AI mask, evaluate the materials you are working with. Low contrast images can be difficult for AI, so it makes sense to use them when the color correction is finished, or prepare a temporary grade on another node. Increased contrast, brightness and saturation will help you achieve better results. Are you all done? Go to the color page and find the icon with a human silhouette in the center panel below and click on it.
Take a look at this new area with the settings shown. These will help you improve your choice. But first, you need to add some dots to the subject to include all the elements needed for the mat. In general, it is recommended to use the “better” mode, especially if you are working with people or use fine details.
Once the mask is generated, refine with additional points or use negative qualifiers to remove unnecessary areas. After that, we want to track it down. Here it is important to find moments when subjects are fully visible (for example, using all their limbs). Go to this moment in the clip and click on the arrow to track back and forth.
We also recommend experimenting with all the sliders and settings to feel how it affects the mask you create. Once the selection is complete, there are a few things you can use, such as accurate color grading and cool text title effects. However, in this article, we focus on quick fixes, so there are plenty of examples of daily applications below.
Blurred content due to AI masking
Faces, license plates, phone screens… all of these can ruin the perfect shot if you don't have permission to view them. (However, if you don't see them clearly, that's fine.) So, to fix this issue, you can drag and drop the appropriate effects on the nodes with previously created AI masks. In Davinci Resolve, such an effect could be a mosaic blur or focus to give some name.
This is the result of the mosaic blur. Small Tip: Check the “Blur Alpha” field in the Effect Settings and the blur will match the outline of the masked area more accurately.
If you want to slightly expand your selection and soften the edges, you can also increase the “Clean White” setting in the mask option. And yes, I know it's not elegant or flashy. However, if you're talking about documentaries or other real-life material where something important happens in the story and you don't have the chance to get additional permission, that's definitely a simple fix.
Delete unnecessary objects
Sometimes it's not the person you have to blur, but a distracting sticker, a trash can, or a colorful sign in the corner where you need to attract and remove all your attention. The most frequent scenario I believe happens to every editor every now and then is when I have to hide random company logos that have nothing to do with the video.
Resolve has the effect of removing objects in combination with Magic Mask, and is also an AI-enhanced feature. This process is rather similar to what we discussed previously, but has some nuances.
- Mask and track the objects to be deleted.
- Applies the object deletion effect to the mask node.
- Press “Scene Analysis” and then “Build a Clean Plate.”
- Finely tune it with a clean white/black slider for smooth edges.
It doesn't always give you the perfect result. In most cases, you can see traces of where the distraction element was. However, in many cases, this workflow is excellent enough that no one notices other masking unless you pause and scrutinize the clip.
roto ai masking
In previous demos, we selected objects for shots and added effects. However, AI masking can also be used to separate subjects, place them on different backgrounds, or use them as track mats for creative overlays. To do this, you will need to right-click on the free space in the node area and select “Add Alpha Output” from the drop-down menu. A blue dot appears inside the tab, simply connect the mask node (drag from the blue box on the side onto this new dot).
When I studied in university, I had a course on introducing VFX. I vividly remember how my classmates rotoscope a person with a shot and then another video layer on top of it (with some natural landscapes) to create an abstract, poetic shot. He had to use a selection process for each frame and manually adjust the mask each time. I thought I couldn't do such a beautiful thing. But now it's possible within two clicks. After masking, tracking, and isolating subjects, click the sphere icon in Settings to invert the mask. Now your subject acts as a living “window” for what you want to show within their silhouette. Add a graphic or video layer under the clip on the edit page and enjoy the results.
AI Masking: Conclusion
Naturally, these are just a few ways to implement AI masking in your editing and color grading workflow. In reality, creative possibilities are endless. We'll explain many additional applications in the course “AI Video Masking” on Mzed.com.
Overall, AI masking is useful for editors working on tight deadlines and small projects. However, remember that it is not a replacement for a trained professional. In my opinion, complex synthesis, high-end visual effects, or carefully crafted lighting should continue to be handled by specialists. Masking of AI should be considered a supportive tool. It speeds up your daily work, expands your creative possibilities and frees up your time for truly important decisions.
What else can I gain as a Mzed Pro member?
As Mzed Pro Memberswith access to hundreds of hours of filmmaking education. Plus, we are constantly adding courses (some are currently in production).
Starting from just $29 a month (you'll be charged for $349 in your first year, $199 in your second year, or $49 a month), you can find:
- Over 60 courses and over 800 high quality lessons.
- Highly produced courses from decades of experience and award-winning educators, including the Pulitzer Prize and the Academy Award.
- Unlimited access to stream all content for 12 months.
- Download and view it using the Mzed iOS app.
- ARRI Academy Online course discounts, Mzed only.
- Most of our courses provide industry-recognized certificates upon completion.
- Buying a full course costs over $9,500.
- Course topics include cinematography, director, lighting, cameras and lenses, production, indie filmmaking, writing, editing, color grading, audio, time lapse, pitch decks, and more.
- 7-day money-back guarantee if you believe it's not for you (annual billing only).
Full disclosure: mzed teeth Owned by Cined.
Join Mzed Pro today and start watching today!
Functional image source:mzed.
I'd love to hear about your experience! How often do you use AI masking in your project (if you've already done it)? In most cases, which of these possibilities is it dependent on? Do you see masking in AI as a safety net or as a creative tool? Let's exchange some thoughts in the comments!
