How well did the AI-generated video reproduce the news article?

AI Video & Visuals


The eerie pile of papers is followed by a gray hallway leading to barred windows. Thirty seconds later, I see a dingy room with warning signs on the walls and a hospital bed covered in tattered sheets.

That’s the method a new short-form video generator has chosen to illustrate a news story about New York State’s treatment of incarcerated people published by City Reporter, a nonprofit newsroom. According to Kennedy Sessions, the article’s lead reporter, the atmosphere was a little “eerie, a little scary.”

QR code for SAN app download

Download the SAN app today to stay up to date with Unbiased. Straight Facts™.

Point your phone’s camera here

Late last month, Notebook LM, Google’s “AI-powered research partner,” rolled out a new 60-second short-form explanation format. The video generator can take any source and spit out a TikTok-style video.

This product aims to condense important information into a more understandable format. As a research assistant, Notebook LM does not search the web. Rather, it is a summary of specific information or text.

Some of the reactions regarding X were enthusiastic.

“No more brain rot and maxed out education,” one social media user posted.

“I’ve always wanted something like this! Super personalized educational tiktok,” another wrote.

But not everyone was a fan.

One user lamented, “In the name of education, a lot of AI leaks will appear on TikTok and Instagram reels.”

So which one is it?

According to Jordan Wilson, host of the Everyday AI podcast, Notebook LM has created the first short-form video generator explicitly aimed at education.

“This is the first time I’ve seen an education-focused AI product come out with vertical video,” he told Straight Arrow. “Because usually those two don’t match up, right?

“This is interesting because AI video often goes hand-in-hand with brain decay,” Wilson added.

The product could be very useful to businesses, he said. One example: Create an onboarding video for new employees who are used to consuming content in video format.

“Many people learn better that way when they can watch interactive content and multimedia, such as text, video, and audio narration,” he said.

But the accuracy of the video depends on experience and effort, Wilson said. Knowing which prompts to give your model can be a matter of understanding what is actually important in the source document.

“It takes a little bit of practice,” he told Straight Arrow.

Does this help you understand the news?

Straight Arrow has used this tool to create several videos from articles published by newsrooms across the country, including our own. Reporters of the original work primarily stated that while they found the videos broadly informative and often accurate, they often missed the nuances of the original story.

In each case, Straight Arrow provided Notebook LM with a link to the original report and asked it to produce a video without providing any guidance as to the content of the video. This reflects how learners would use the tool if they have not yet read the story.

The situation is as follows.

The cost of prison abuse: $25.7 million in New York settlement

Outlet: City Reporter

Reporter: Kennedy Sessions

The video is controversial, Sessions told Straight Arrow. Sessions liked that the article “gives people a short summary,” but noted that the information did not accurately reflect the original reporting.

“It was a little scary,” Sessions told Straight Arrow. “There was a sound, and I don’t know why, but the sound wasn’t like when I was reading an article, it was like an automated voice was reading the article to me.”

And she said some of the facts in the video were wrong. Although the video described one of the characters in the story as being 20 years old, he was actually 20 years old at the time of his incarceration.

Sessions believes that journalism serves an important educational role, teaching people that they deserve accountability from governments and institutions. She believes this type of video creates an educational sound, but manages to exclude any discussion of responsibility for the harms experienced by incarcerated people.

Sessions was a data reporter and charted original works not included in the story. She also felt the video didn’t make it clear that the story was based on real news, and many important voices were left out, she said.

“In some ways, it feels too robotic,” Sessions said. “I think part of certain investigative work is the human connection and the human part.”

This is especially important in journalism and storytelling. “Google Video can’t quantify that.”

Trump administration cuts spending on solar power. Some Delaware farmers are celebrating

Outlet: Spotlight Delaware

Reporter: Olivia Marble

The 72-second video version of Olivia Marble’s article for the nonprofit news station Spotlight Delaware missed her reporting, Marble told Straight Arrow. The written story about the government ending the federal solar financing program included some perspective that was notably absent from the AI ​​video version.

“When I saw that video, I was honestly a little disappointed,” she said. Because this video turned one of the several perspectives she included into the dominant narrative.

“It didn’t show any of the subtleties of what I was trying to convey,” Marble told Straight Arrow.

As a reporter, Marble strives to include all sides of any discussion. She said she felt the AI ​​did a good job of explaining one side of the argument, but narrowed the focus to the exclusion of others.

“It didn’t include any of the objections I put in there,” she said.

And, she says, the video was completely taken off the “news peg.” “There is no mention of the Trump administration taking away the loans,” she said.

Ultimately, she said, the platform turned her journalism into opinion pieces, or “edutainment.” She doesn’t expect everyone to always read the full article. She said they might just happen to read the headline because they’re busy. But this kind of video summary doesn’t give you the whole picture.

“My fear is that someone will look at this and give the false impression that they understand everything I’ve written,” Marble said.

Dover’s begging controversy re-emerges as flashpoint for city issues

Outlet: Spotlight Delaware

Reporter: Maggie Reynolds

Maggie Reynolds, also of Spotlight Delaware, reported on the experience of local communities debating proposed panhandling ordinances. She found that the video made language choices that overgeneralized her reporting and exaggerated the experiences of her sources.

Reynolds strives for precision in his work. So when she watched the Notebook LM video, she was disappointed to find that some of the analysis video contradicted the information that was included in her story.

“I know it’s like a 60-second video, so it’s shorter than a full-length novel, but it had a lot of general remarks,” she told Straight Arrow.

Another check on accuracy: The video chose words she didn’t use to describe the source’s actions, depicting residents as “aggressively” and “desperately” pushing for the new ordinance.

“In reporting, it’s important to choose your words accurately,” she told Straight Arrow. “When using such strong language, be sure to use accurate details and evidence in the story.”

In this case, she said, that’s not the case.

Like her colleague Marble, Reynolds noted that her story includes a wide range of voices. But this video focused on just one person.

“We write one-minute scripts for social media, and we know it’s hard to put a lot of depth into them,” Reynolds said.

If she were to create a version of this video, she said, she would provide more context about specific discussion flashpoints and incorporate more voices on the ground.

These ranchers heal the land. The power company drew the line there.

Exit: Straight Arrow

Reporter: Keaton Peters

Keaton Peters spent two months reporting on Texas ranchers concerned that a proposed new power line would run through their property, impacting their ability to use electric fencing to move cattle.

Peters said the video did a great job of explaining parts of a complex story.

“I was a little impressed that the AI ​​understood that part,” he said. “But there were some important things that were lost when we focused solely on that.”

The first is attribution. While the video did a good job of elucidating one of the technical issues at the heart of the story’s conflict, “that wasn’t the point of the story,” he said. The video did not show that the power companies involved disputed this point of view, nor did it point out why the grid was being developed in the first place.

“It didn’t have the nuance of who was saying it, so it looked like it was a universal truth, rather than the experience of people on one ranch,” Peters says.

Peters said this not only made the video inaccurate but also obscured the overall picture, “upsetting the balance of making sure to show the perspective of everyone involved.”

Peters said it would be difficult for a human to condense an article that took two months to report into one minute. But it was a grave mistake to miss the central point: it was about land rights, not the power grid.

A blogger sent a photo of Shrek’s penis to a congressman. He now faces a six-month prison sentence

Exit: Straight Arrow

Reporter: Mark Kierever

It was a simpler story for Keareber, which focused on the First Amendment implications of a blogger who texted a lewd photo of the animated character “Shrek” to a state senator.

The video tells the story well, Kierever said. However, he had some problems with it hallucinating new elements or misrepresenting the facts of the story.

“What’s really interesting is that the AI-generated narrative says, ‘Not only did the politician block the blogger, but he also filed a formal complaint,'” Kierever said, though he never mentioned “blocking” in his story.

This is a classic example of LLM’s use of the well-known “It’s not X, it’s Y” rhetoric. In this case, it created a hallucination that viewers could interpret to mean something different than what actually happened, Kierever said.

His main problem with the video is that the Supreme Court has a three-pronged standard for determining whether something is protected by the First Amendment.

“This video actually to some extent oversimplifies what is a fairly important and bureaucratic legal standard, as it only really highlighted the first of the three aspects,” he said. If viewers rely solely on video, they risk losing a complete understanding of the law.

Additionally, some of the images included fabricated shots of state troopers, and Kierever said, “None of these images are authentic and may be misleading to viewers.”

The tone of the image also seemed to be lowered.

“This stark black and white footage read like a really grueling story,” Kierever said. “In this case, on the other hand, we are not talking about a particularly serious issue, but we are certainly talking about an important issue: the First Amendment.”

The video also mentions two law enforcement officers arresting the blogger. That’s new.

“In fact, I never mentioned in my story that it was two people who arrested him,” Kierever said. “The question arises whether it was an illusion or whether the video generator actually obtains content from other sources that exist on the Internet.”

What can we learn from these videos?

“This is new technology,” Wilson said. “Google is pushing the boundaries of what’s available.”

However, it is still up for debate whether AI-generated vertical videos can be a useful compositing tool for education. Reporters agreed: While the Notebook LM video captures some of the source material, it seems to miss the nuance and purpose of the reporting.

“If we want to reach more people in a more digestible video format, we want to put our resources into creating videos like this from our own work rather than something generated by AI,” Spotlight Delaware’s Reynolds told Straight Arrow.


Finish your reading



Source link