Runway recently launched their Gen-3 Alpha tool, taking AI-powered video production to the next level. By the end of this guide, you'll have a clearer understanding of how to use Runway's Lip Sync tool and be able to decide if it's the right solution for your video production needs.
As one of the original developers of the groundbreaking Stable Diffusion image generation platform, Runway brings extensive experience to all aspects of AI-powered image and video creation. One of Runway's most easy-to-use and popular features is its audio-to-video lip sync tool. This technology allows users to create lifelike talking videos by matching lip movements to audio input. Whether you're a content creator, educator, or business professional, this tool offers a variety of applications to enhance your video projects.
1. Log in
(Image: ©Future)
Log in Runway. If you are new to the AI platform, you can sign up using your Gmail account.
2. Interface Operation
(Image: ©Future)
The Video Lip Sync feature can be found under the Audio menu on your Runway account dashboard. It also has 3D tools, image and video generation. Choose your preferred voice From the options.
What's great about the user interface is the liberal use of tutorial material at every step, which is reassuring for beginners starting their AI media journey.
3. Enter the prompt
(Image: ©Future)
The Lip Sync page itself is very simple and easy to use. The top left is space for user input. This can be done via a text prompt, an uploaded audio file, or a tool to record audio on the fly.
If you upgrade to the Pro plan for $35 per month, you can also record your own custom voice to use with your avatar.
4. Choose your voice
(Image: ©Future)
I decided to upload an AI-generated avatar I had created previously and pair it with a short script to test the accuracy of the lip movements.
There's a wide variety of voices to choose from, likely a reflection of Runway's history of working closely with the film industry, so there's sure to be something for everyone.
5. Generate the video
(Image: ©Future)
Once the auditions and voice selection are over, Upload your script (or a pre-recorded audio clip);[生成]Press the button.
The free plan generates requests at a slower pace than the paid plans, so your request may have to wait in the queue for a few minutes before being fulfilled. The whole process takes about 3-4 minutes It depends on the length of the video and the server load at the time.
6. Understand your credit usage
(Image: ©Future)
Text-to-speech uses 1 credit for every 50 characters of text converted to speech, and lip sync uses 5 credits for every second of audio.
Both costs are added to the total for the video, so essentially my 38 second video used about 250 credits. 1 credit is worth $0.01 USD. Standard plan users get 625 credits per month with a $15 subscription.
result…
Skip to 3.25 to see the final product below.
Once the video generation is complete, you can preview the result on the right side of the webpage. For free users, all videos will have a watermark, but at least it's not too ugly.
I was also surprised at how good the actual video results were with the images I uploaded, with accurate lip movements during speech and some matching facial movements.
Although the overall effect is not perfect and there is still a bit of “uncanny valley” effect, it is quite suitable for general business needs. After creating the video, you can enjoy various options, such as downloading, regenerating, editing the script and other elements.
Overall it was a quick and easy process and the results are truly top notch, as you would expect from a company with this proven track record.
There are many similarities between Runway and Pika Labs, another prominent player in the AI video generation space. In fact, Pika Labs has a similar lip-sync feature, also using ElevenLabs' natural voice. There are many AI video generators out there, but the best ones produce amazing results.
