Can you imagine seeing yourself in your favorite show without ever actually acting? Or is it possible to change the beginning and end at will?
San Francisco startup Fable Studios just released SHOW-1 AI technology that can write, produce, direct, animate, and even create voices for all-new episodes of TV shows. Fable Studio did it using various diffusion models. They operate in a simple way that adds and removes random noise to the data over time and can generate and reconstruct the output. You can start your image as random noise and gradually transform it into the desired output.
Fable Studio trained a diffusion model over a dataset consisting of 1,200 characters and 600 background images from the TV show South Park. Their first model task was to generate a single letter he against a background color. Autonomous characters can be generated within a show based on the person’s distinctive appearance, writing style, and voice. The Character Diffusion Model allows you to create a South Park character based on your appearance through steady diffusion between images.
🚀 Build high-quality training datasets, solve NLP machine learning challenges, and develop powerful ML applications with Kili Technology
The second model’s task was to generate a clean background that acted as a stage for the characters to interact with, allowing multiple scenes and scenarios to be designed. The only limitation of this model is the low resolution of the images produced. The team tackled this problem using an AI upscaling technique that improves image quality. Produces vector-based graphics as it does not lose resolution when rescaling.
Fable Studios redefined the television show’s episodes by changing the order of dialogue in certain locations and running times to match the original length of the episode. Using simulation data as a prompt chain, we built a story system that ran parallel to the showrunner’s system, overseeing the action and dialogue sequences to keep the audience engaged. Each character’s voice is pre-cloned and a voice clip is generated for each new dialogue.
The data generated by the simulation serves as a creative dictionary for both the individual creating the initial prompts and the story system generated. Even experienced story writers often get stuck when writing dialogue. Such problems can be overcome because the simulation provides context and data points before starting her chain of prompts.
The story generation process is shared proportionally between the user, the simulation and GPT-4. A simulation generates a basic context of a character’s history, emotions, events, etc. This serves as your initial creative context. GPT 4 acts as the main generation engine, merging scenes and dialogue based on prompts and simulations received from the user.
Finally, the best of simulations, users, and AI models come together to create richer, interactive, and engaging storytelling experiences. On the contrary, personalizing the show leads to job losses. The ability of AI-powered tools to perform tasks once performed by human experts, such as video editing and music composition, will lead to concerns about the future of jobs in the entertainment industry.
Please check plan and Reddit post. All credit for this research goes to the researchers of this project.Also, don’t forget to participate 26,000+ ML SubReddit, Discord channeland email newsletterShare the latest AI research news, cool AI projects, and more.
🚀 Check out 900+ AI Tools in the AI Tools Club
Arshad is an intern at MarktechPost. He is currently continuing his international studies. He holds a master’s degree in physics from the Indian Institute of Technology, Kharagpur. Understanding things from the ground up leads to new discoveries and advances in technology. He is passionate about using tools such as mathematical models, ML models, and AI to understand the fundamentals of things.
🔥 Gain a competitive edge with data: Actionable market intelligence for global brands, retailers, analysts and investors. (with sponsorship)
