This article first appeared in China Report, MIT Technology Review's newsletter on Chinese technology. Sign up to receive it by email every Tuesday.
You may not know much about Kuaishou, but the Chinese company just hit a major milestone: it released the first text-to-video AI model that the public can test for free.
The short-form video platform, which has more than 600 million active users, announced a new tool on June 6 called “Kling.” Similar to OpenAI's Sora model, Kling can generate videos “up to two minutes long at a frame rate of 30fps and video resolutions up to 1080p,” according to the company's website.
Don't be satisfied with half stories.
Get access to the latest tech news with no paywalls.
Subscribe now
Already a subscriber? Sign in
But unlike Sora, which OpenAI has yet to release to the public four months after it was piloted, Kling quickly began allowing people to try out the model for themselves.
I was one of them. I was able to access the model by downloading Kuaishou's video editing tool, registering with a Chinese phone number, joining the waiting list, and filling out an additional form through Kuaishou's user feedback group. The model can't handle prompts written entirely in English, but you can get around this by translating the phrase you want to use into Chinese or including a Chinese word or two.
So, first things first. Here are some results we generated with Kling to give you an idea of what it looks like. Remember Sora's impressive demo video of the Tokyo streets and a cat running through a garden? Here's Kling's take:
ZEYI YANG/MIT Technology Review | Kling
ZEYI YANG/MIT Technology Review | Kling
ZEYI YANG/MIT Technology Review | Kling
Remember that image of the astronaut on the horse in Dall-E? I asked Kling to create a video version as well.
ZEYI YANG/MIT Technology Review | Kling
There are a few things to praise here: none of these videos stray too far from the prompt and seem physically correct, with camera pans, rippling leaves, and the way the horse and astronaut turn to reveal the Earth behind them. The generation process took about three minutes each — not the quickest, but absolutely fine.
But there are some obvious drawbacks: the videos, although in 720p format, look blurry and grainy, Kling sometimes ignores key prompt requests, and most importantly, all videos currently generated are limited to a length of 5 seconds, making them significantly less dynamic and complex.
But it's not really fair to compare these results to something like the Sora demo that OpenAI has handpicked and made public, and which arguably shows above-average results. These Kling videos were my first attempts with each prompt, and I included few prompt engineering keywords like “8k, photorealism” to fine-tune the results.
If you want to see more videos generated by Kling, check out this handy collection compiled by China's open source AI community, which includes both impressive results and all sorts of failures.
Kling's general capabilities are good enough, says Guizhan, an AI artist in Beijing, who has been testing the model since its release and putting together a series of head-to-head comparisons between Sora and Kling. Kling's shortcomings, he says, lie in the aesthetics of the results, including composition and color grading. “But it's not a big problem, and you can fix it quickly,” says Guizhan, who asked to be identified only by his online alias. MIT Technology Review.
“The core functionality of the model is the laws of physics and how it simulates the real natural environment,” he says, and in that respect Kling excels.
Kling works in a similar way to Sora, combining a diffusion model traditionally used in video generation AI with a transformer architecture to make sense of larger video data files and produce results more efficiently.
But Kling may have a key advantage over Sora: Douyin's biggest rival in China, Kuaishou, has a massive video platform with hundreds of millions of users uploading vast amounts of video data that can be used to train Kling. “Kling follows industry standards and uses publicly available data from the internet around the world to train our models,” Kuaishou said in a statement to MIT Technology Review, but the company declined to provide details about the training data (OpenAI also declined to provide details about Sora, raising concerns about intellectual property protection).
After testing the model, we felt that the biggest limitation to Kling’s usefulness was that it could only generate five-second videos.
“The longer the video, the more likely it is to cause hallucinations or produce inconsistent results,” says Shen Yang, a professor of AI and media at Beijing's Tsinghua University. This limitation means the technology will have a bigger impact on the short-video industry than the film industry, he says.
Short vertical videos (made to be viewed on smartphones) typically grab a viewer's attention within a few seconds. Shen said platforms like China's TikTok often judge a video's success by how many people watch the first three or five seconds above the fold. So high-quality AI-generated video clips of just five seconds could be a game-changer for short-form video creators.
Keizo agrees that AI could disrupt the short video content creation scene. In the short term, AI will benefit creators as a productivity tool, but in the long term, he worries that platforms such as Kuaishou and Douyin could take over video production and directly generate customized content for users, reducing the platforms' reliance on star creators.
While it may still be quite some time before the technology reaches that level, the field of text-to-video creation tools is now heating up considerably. A week after Kling's release, California-based startup Luma AI released a similar model to the public. Runway, a popular video generation startup, has teased a major update that will make its model even more powerful. Kuaishou's biggest rival, ByteDance, is also reportedly planning to release a video generation tool soon. “There will be a lot of options available by the end of the year,” Guizang says.
I asked Kling to generate a vision of what a society where “everyone can quickly create video clips based on their needs” would look like. And here is the result. It's a great move, but it doesn't answer the question. Sorry.
ZEYI YANG/MIT Technology Review | Kling
Have a prompt you'd like to generate with Kling? Submit it! email address We will reply with the result. Please make sure your prompt is less than 200 characters long, preferably in Chinese.
Catching up with China
1. A new investigation has found that the U.S. military ran a covert campaign of anti-vaccine propaganda posts on social media in 2020 and 2021 to sow distrust in Chinese-made COVID-19 vaccines in Southeast Asian countries. (Reuters$)
2. A Chinese court sentenced Huang Xueqin, a journalist who helped launch the #MeToo movement in China, to five years in prison for “inciting subversion of state power.” (Washington Post)
3. A Shain executive said the company's valuation essentially makes it an American company, but the company is now trying to keep that statement secret so as not to anger Beijing. (Financial Times)
4. China is close to building the world's largest particle accelerator, with construction possibly starting as soon as 2027. (Nature)
5. In retaliation for the European Union's increased tariffs on electric cars, the Chinese government has launched an investigation into alleged unfair subsidies for European pork exports. (The New York Times)
- Related food info: China's demand for durian has exploded in recent years, creating a $6 billion business in Southeast Asia and prompting some farmers to cut down jungles and coffee trees to make way for durian plantations. (The New York Times)
Lost in Translation
In 2012, Jiumei, a Chinese woman in her 20s, began selling a service to send “good night” text messages to people online for 1 RMB (about $0.14) per message.
According to a Chinese online publication, 12 years, three cell phones, four different numbers and more than 50,000 messages later, she's still at it. personSome of her customers buy the service for themselves, hoping to have someone to talk to on a regular basis when they're at their loneliest or most desperate, while others buy it to send anonymous messages to friends going through difficult times or exes with whom they've lost touch.
The business isn't particularly lucrative — Jiumei makes about 3,000 yuan ($410) a year from it on top of her day job, but it's fallen even further in recent years — but she continues to do so because sending these messages has become a nightly ritual that brings solace not only to her clients, but also to her own personal moments of loneliness and hardship.
one more
Globally, Kuaishou has not been as successful as rival ByteDance, with the exception of Brazil, where Kuaishou's international version, Kwai, is so popular that even the Marbo people, a tribe living deep in the Amazon rainforest and one of the last connected communities online, have started using the app, according to Brazilian media. The New York Times.
