Over the past two years, developments in artificial intelligence (AI) have sparked a wave of amazing innovation across a variety of sectors. For app developers, this comes with the concurrent challenge of ensuring AI technology is reliable, effective, and safe. AI capabilities are constantly evolving, so we must constantly be on the lookout for more we can learn, integrate, or develop ourselves.
For Lightricks, harnessing cutting-edge AI has been both exciting and challenging. After integrating dozens of new Generative AI (GenAI) features into our flagship visual content creation apps Videoleap, Photoleap, and Facetune, over the past year we took on a new challenge to develop LTX Studio. LTX Studio is a filmmaking app that pushes the boundaries by integrating AI into filmmaking using GenAI. It allows users to transform a single idea into a cohesive, AI-generated video, amplifying creativity through new forms of storytelling.
Our team has several years of experience working on visual AI apps, but in many ways this project crossed uncharted territory – not just for us, but for the entire tech industry as well. Researching, building, and refining the app involved staying on top of the rapidly evolving AI landscape, identifying potential technologies we could integrate or develop in-house, and orchestrating it all with creative problem solving. We're proud of what we've accomplished with this product, but our work isn't done yet.
Here are some of the things we've learned so far.
Build your models with care and intention
Every generative AI tool is limited by the quality and scope of the data behind it. Because of this complexity, our researchers and engineers go to great lengths to ensure our models are consistent and reliable.
It's also important to pay attention to possible biases within our AI models: our engineers are constantly refining how the engine interprets user prompts to ensure the output is unbiased and as close as possible to the user's vision.
Taking Usability Testing to a New Level
AI tools tend to be highly technical and run the risk of being too complex for the average user to apply effectively. We needed to wrap LTX Studio in a user-friendly, accessible interface with features, actions, and terminology already familiar to video creators. Plus, we're working directly with leaders in the filmmaking and advertising industries to create the capabilities they need to meet their ever-changing challenges.
We've always had a user research team and have taken a data-driven approach to iterating on our products based on user interaction data, but now we've added a beta community to the user feedback mix, hubped in a Discord server, with active members posting and sharing their feedback on Discord and providing feedback directly to the CX team. Their feedback has been invaluable and helped us refine the interface significantly.
Based on past project experience
Ensuring that AI tools are stable and scalable is difficult, especially when things are changing rapidly, and even more so when you're breaking new ground in AI.
Luckily, we had experience building similar software before.
Moreover, from previous development projects we already knew a lot about the challenges we would face.
Balancing the need for speed and quality
The challenge with creating generative AI content, especially as we move into the world of generative video, is that rendering and creating high-quality, high-resolution video content is complex and time-consuming. Rendering video faster typically comes at the expense of quality, but keeping the user engaged while maintaining a smooth editing experience is paramount.
How did we achieve this? First, we break the production experience into small parts and guide the user through the experience, moving scene by scene, shot by shot. Since there is no need to wait for the entire movie to be rendered, the project can be built incrementally. Second, we build a rendering process that works incrementally, allowing the user to make faster decisions about the final result. This way, the user can choose between different versions during the creative process (avoiding suboptimal, low-quality video) and wait only for the relevant result.
Multi-tenant scalability
AI video is compute-intensive, and even with faster rendering, it's still difficult to scale an AI video editing platform to many simultaneous users without buffering or long wait times.
Managing global surges in mobile app usage, our engineers have established best practices and adapted quickly to maintain the best user experience.
Bringing GenAI Video to Life
Creating a generative AI video platform is an ongoing yet exciting challenge. At Lightricks, we are constantly striving to find new ways to integrate AI into our innovative tools. Learning from experiences across our portfolio, from editing app Facetune for portraits, to Photoleap for images, to Videoleap for video content, allows us to integrate new elements and experiences for our users.
Building LTX Studio has made it clear how important it is to continually interact with our user community to ensure that technology is evolving in the right direction to continue to meet our users' needs and standards. We're excited to see what the future holds.
Alon Yaar is VP of Product at Lightricks, an award-winning company that develops AI-first photo and video editing tools that have been downloaded more than 730 million times worldwide. An experienced freelance designer, Yaar joined the Lightricks team in 2018 as a Machine Learning Researcher.
