A New Kind of Revenge Porn

Note: This story will, at times, seem explicit. We intend to inform the reader (that’s you) of the most important human stories from the AI sector. This is an important story that will have long-lasting impacts on society.

One of our motos at Subscribe to AI is “AI will never be this bad again.” We stand by this sentiment. As time passes, the technology will get better.

This has enormous implications across many sectors of the economy and in our daily/ personal lives. But it is also important to note that AI is a tool that can give a voice back to stroke victims, a tool that can uncover lost artefacts from the most famous volcano eruption.

AI can help end malaria and detect early lung cancer. But like any tool, it can be misused. This story is about that. What we are about to describe isn’t an indictment of the tool used - but of the people wielding it. 

CivitAI is a website dedicated to sharing AI-generated images. Venture capital group Andreesen Horowitz says it is now the seventh most used AI tool. 

There are thousands of AI models that can create new and original images. For example, if you find an AI that is good at generating images of cars and one that generates images of fruit, you can make an image of pineapple cars in traffic. 

This is the part of the story where we discuss how AI is used. In this case, it isn’t about solving cancer or creating children's books. Nope- it's used to create pornographic images.

So, on the site, the most popular sections on the website are pornographic images and celebrities.

And although the official terms of service of the website are you can’t make pornographic images with real people - it hasn’t stopped it from happening. It doesn’t take long to find AI-generated images of famous celebrities in compromising positions.

This can be anywhere from apparent nude photographs to being in hardcore pornography. And even when they are taken down- it doesn’t stop users from explaining how to do it off the website. Meaning these images are often posted elsewhere. 

Let’s take a step back and reevaluate what I’m saying: people are taking images of sex workers that are online (without consent), using them to train AI Image Generators, and then taking pictures of non-consenting personnel (celebrities, influencers, and regular people) and creating AI porn of them. 

This is just a continuation of the problem deepfakes have presented in the past, but it is happening at a rate so fast that it is almost impossible to keep up with. While most of us can agree that this practice is entirely immoral, it only worsens. People are creating business models based on this practice.

In addition to the individuals using Patreon and other services, CivitAI created the option to offer “bounties” to make AI renderings of specific people. Whoever does the best gets virtual currency from the website. 

As you can expect, bounties for celebrities and social media influencers started flooding the pages. Also unsurprising - the bounties were placed almost exclusively on women. In one case, a person uploaded low-resolution photos of his “wife” and wanted an AI model made of her.

She’s not a celebrity, and as users of the site found out after finding her on social media, she was not his wife. Despite the scepticism towards the post, someone made an AI model of her to collect the bounty.

Although most bounty requests aren’t explicit, most agree that CivitAI has monetized something that is already happening on their website. And with a lack of actual measures to enforce the policy around non-consensual pornography - these issues will only continue to spring up. 

While we wait for the legal protections to catch up to AI, CivitAI just got a huge paycheck from the legendary venture capital group Andreessen Horowitz.

Yep, the same group evaluated 50 AI websites and ranked it 7th. 

CitiviAI received over $5,000,000 in this fundraising round.

Maybe the investment will put the proverbial adults in the proverbial room.

It doesn’t appear to be that way, though.

Marc Andreessen (half of the venture capital group's namesake) posted a manifesto about his belief in “techno-optimism.” He suggests that any deceleration of AI is a form of murder. So, he might not see the regulation of non-consensual AI-generated porn as an issue he is willing to slow down for. 

There’s not a great way to end this story. People are using AI to make pornographic images of regular people who have given zero consent for those images to exist. Strangers online are making pornographic images of other strangers for virtual currency as a bounty. It is purely transactional with disregard for the potential impacts.

AI will never be this bad again - we can only hope this is as bad as humanity’s use of it gets, too.