The Senate Passes the DEFIANCE Act

The DEFIANCE Act is a win. This doesn’t mean that AI is free from sexual exploitation. 

Assuming the House takes up the DEFIANCE Act, victims of non-consensual sexually explicit deep fakes will soon be able to sue those who create and distribute images. This is a win for women and girls, who are disproportionately targeted by such harassment. The DEFIANCE Act places power back into the hands of victims and should be applauded accordingly. However, we shouldn’t delude ourselves into thinking we’ve fully confronted the complex intersection between AI and sexual exploitation. 

There’s very few people who would disagree with the DEFIANCE Act. The proliferation of deepfake technology means that teenage girls and celebrities alike have seen themselves in sexually explicit content they never agreed to. In one instance, a young college student found AI-generated videos of herself and her personal details on Pornhub, in what seems to be the bitter actions of a disgruntled male classmate. When reflecting on her experience, she noted that it was “surreal seeing my face … especially the eyes…They looked kind of dead inside” 

Such acts are deeply malicious and traumatic. Victims have mentioned a sense of powerlessness, since there is very little they could have done to prevent these videos. Students face bullying from their peers or potentially irreparable damage to their careers. While these repercussions may never eventuate, paranoia still seeps in as victims fixate on which strangers are staring at them or who may have seen this content. All of this distracts from living a healthy life, pursuing a fulfilling career or education. Thus it’s a relief that the DEFIANCE Act is finally here, albeit slightly late. 

Yet this step forward doesn’t mean that AI is free from sexual exploitation. Perhaps we should ask ourselves how these tools can even generate sexually explicit imagery, including that of children. The answer is training data. 

Sadly, many image generation models are trained on images and videos of abuse. Stanford researcher David Thiel recently identified hundreds of known child sexual abuse materials (CSAM) in a dataset used to train Stable Diffusion. For those questioning if there’s additional harm when these images have already been generated, the fear is that any additional demand for CSAM increases the rates of such abuse. While the content is currently being removed from this specific dataset, this example highlights the need for companies to thoroughly understand what data they’re using to train their models. 

Children aside, there’s plenty of other moral considerations when it comes to generating sexually explicit content. AI models that generate porn are almost “certain” to have been trained on non‐consensual intimate imagery (NCII), sometimes accidentally, sometimes not. For example, an image generator called ‘These Nudes Do Not Exist’ has been linked to Czech Casting, a porn production company under investigation for human trafficking and rape. Women who were coerced into sex often don’t want that footage to spread, let alone used to train AI models. In the words of one victim, "it feels unfair, it feels like my freedom is being taken away".

Even without using data that depicts abuse, AI-generated explicit content is ethically dubious. Short of banning the word ‘child’ from prompts, it can be hard to prevent tools from generating CSAM. And CSAM can theoretically be generated from otherwise innocuous images of children. Are we okay with that? How might the parents of children whose images are used feel? 

What about if a young woman posts photos with her friends on the beach to social media? Should she get a say in whether those pictures are used to train AI-generated porn? 

Finally, there’s the question of compensation and copyright. Porn generation tools directly profit off the men and women who featured in the training data. Some of these people may have consensually been in porn; others may just have fully-clothed pictures of themselves on the internet. Among those who don’t want to “opt-out”, some people might reasonably expect to be compensated for the use of their likeness. 

These are contentious topics.“Opt-out” for social media may be a logistically tricky issue. Not everyone will agree that there should be compensation for those who have consensual images in training data. I personally don’t think that AI should generate CSAM, regardless of how innocuous its training data was, but some may argue that it’s a ‘victimless crime’. At the very least, I think we can all agree that AI companies should not be using abusive content to train their models. 

Companies should be taking as many steps as they can to understand the provenance of their datasets and prevent accidental inclusion of abuse material. To start with, the NIST AI Safety Institute could provide best practices on how to avoid this. Ideally, we would get to a point where there are clear legal incentives to adhere to such practices. 

This is not a crazy request for companies. After all, they really should know what is going into their models. If an AI developer can’t keep CSAM out of its training data, then how can they say with a straight face that their AI system is safe? And if they can’t offer reasonable assurances that the system is safe, why should they be allowed to deploy it?