Waikato Otaua School principal Anna Voyce was also a victim of these videos, but she knows it could have been much worse.
“They found a photo of me and made some TikTok videos with my photo and made it look like I was talking, moving my mouth, moving my facial expressions… so it was believable.”
In the video, she was shown saying bad things about certain students and making disparaging comments about the school.
“It certainly brings some stress because of how real it looks and how believable the video is…The anxiety and worry comes with it, not just for myself and being portrayed in such a light, but also the negative impact it has on our school.
“Our school is a warm and caring place, and given what’s on TikTok, what I most wanted to convey to our community and the wider world was the false statement that appeared to come from my mouth.”
In some ways, Boyce said it was a relief to know that things could escalate further.
Mr Walsh said the cases he reported were extremely distressing.
“It’s basically a teacher in a sexual pose without any clothes on.
“Some of them are clearly fake, and others take a little longer for technicians to detect.”
Walsh said students often don’t understand the devastating impact on teachers.
“This leads to a loss of trust and confidence in the student. It damages the student’s reputation and leaves the student very angry and frustrated.”
Sean Lyons, NetSafe’s chief safety officer, said this had been a problem for many years, but the sophistication and complexity of technology had increased the risk.
“They went from being cartoon-like pieces to something that when someone looked at them, they looked very realistic.”
He said this type of damage is difficult to quantify but can cause significant emotional and psychological distress.
“It can even get to the point where you don’t want to go to work or attend social events because you’re afraid of the repercussions and comments that others who see it may have.”
Lyons said there was also a sense of fear among teachers about what was going on and that they might be targeted.
AI videos and deepfakes are illegal under the Harmful Digital Communications Act. You can order the host site to remove your content.
This is civil law, not criminal law.
However, police will intervene for the most serious criminal offenses, such as threats to safety or the unauthorized sharing of intimate footage.
Voyce said she reported the video to TikTok before seeking support from Netsafe.
She said it had been online for about 12 hours when she saw it and it took about 48 hours before it was removed.
It was a surprise for Voice to be the subject of one of these videos, considering Otaua School’s students range from grades 1 to 8.
“I think this is also a danger of children accessing social media at a young age.
“They generally don’t have the maturity at that point to make good choices consistently all day, every day.
“And just one or two errors in judgment over a short period of time can have significant negative consequences.”
As a result of the incident, Boyce removed all photos of staff members from the school’s new website, fearing it would happen again.
“We have also stopped using images of children on our Facebook page, which we previously preferred, because we are also concerned about the possibility of children appearing in AI videos that they did not consent to.”
Walsh said he’s heard about student-made videos of teachers, but it’s far more common to hear about deepfake videos made by students about other students.
“It is certainly a growing problem area, and it is often boys who are creating these images.
“And for teenage girls, it’s completely destructive to their self-esteem and image.
Walsh said it’s difficult to quantify how many videos of this type occur.
“However, I have spoken to at least five or six principals over the past six months to discuss this topic.”
He said this is a prime example of why social media bans for under-16s are necessary.
“The general public and parents don’t understand that what lies behind social media can be extremely destructive to the mental and emotional health of teenagers.
“Prohibition is not a silver bullet, but it would be an important tool in the fight against it.”
The UK government has announced that it will mirror Australia’s ban and take effect from early next year.
– RNZ
