Last fall, the parents of students at Westfield High School in New Jersey received a disturbing message from Principal Mary Asfendis.
“I am writing to make you aware of a situation that resulted in widespread misinformation and resulted in significant worry and concern amongst the student body of Westfield High School,” Asfendis wrote, according to reports in local media. “Earlier today, students brought to our attention that some of our students had used Artificial Intelligence to create pornographic images from original photos. There was a great deal of concern about who had images created of them and if they were shared. At this time, we believe that any created images have been deleted and are not being circulated.”
The incident resulted in some of the victims’ parents pursuing legal action, and edtech experts say it should serve as a wake-up call to districts across the U.S. After all, the threat of students using a photo of classmates to create AI-generated deepfakes of them engaged in pornographic acts is no longer hypothetical — nor are other potential disruptive uses of AI deepfakes in the school setting.
“I’m concerned that schools are getting caught off guard by the rising use of AI to deepfake,” says Doug Levin, co-founder and national director of the K12 Security Information eXchange, a nonprofit dedicated to protecting K-12 schools from emerging cybersecurity threats. Levin adds that while much of the conversation around AI has focused on its potential and drawbacks in the classroom, he’s worried about adversarial AI use “creating all sorts of havoc in school systems because [they] haven’t ever considered whether or not something like this can happen.”
Chris Aviles, an educator, has similar concerns. “The reality is every school district is ten minutes away from potentially something like this rocking their school community,” says Aviles, who also is the president of Garden State Esports.
Here is what you need to know about the threat of AI deepfakes in the school setting, including deepfake porn.
Deepfake AI and Potential For Problems
Aviles says the incident at Westfield High School drove home the extent of deepfake threats. “The reality is, I don’t think folks generally know how easy that is to do with AI,” he says. “We’re at the point where it takes absolutely no talent. You do a Google search, you have a picture of somebody, you upload it, and it spits out pictures and in some cases actual videos of the person. We see it happening to celebrities. And we see it happening to movie stars.”
Deepfake porn of students generated by other students is perhaps the most disturbing use of the technology, but it has the potential to be disruptive in other ways. In the past, school social media accounts have been hacked to put words in the mouths of school leaders saying things they never said, Levin says, but AI is amplifying these more traditional-style pranks.
“When it’s an actual video of the superintendent, or what appears to be the superintendent, that’s a whole other thing,” Levin says. “I think schools really need to understand this is coming and figure out how they’re going to respond and set expectations and disciplines around this. I’m not sure right now the rules ever foresaw anything like this happening.”
Planning For Deepfakes
Aviles agrees with Levin that the first step in mitigating the threat from deepfakes is awareness.
“The best way to prevent it is to get out ahead of it and make sure everybody knows that, ‘Hey, we know this is a thing that could happen, and here’s what’s going to happen if it does,’” he says.
This includes making sure students are aware of the consequences of creating harmful deepfakes, including criminal charges that can result from creating deepfake porn.
“This should be taught in health classes or technology or digital citizenship classes [so students] understand, No. 1, what happens to the victims of this,” Aviles says.
Next, all students should be educated about the intense harm making these types of deepfake videos can also cause the creators, and how in some instances producing these can result in potentially life-ruining lawsuits and criminal charges, including possession of child pornography, Aviles says.
Finally, Aviles says that legally many cases of deepfake porn are prosecuted with revenge porn laws, however, school leaders should consider creating their own policies addressing their response above and beyond legal ramifications, if they haven’t already. \
“Beyond referring it to police for criminal charges, in my mind this is worthy of expulsion,” he says.