Whether of politicians, pop stars or teenage girls, sexualised deepfakes are on the rise. They hold a mirror to our sexist world

Shutterstock

Victorian MP Georgie Purcell recently spoke out against a digitally edited image in the news media that had altered her body and partially removed some of her clothing.

Whether or not the editing was assisted by artificial intelligence (AI), her experience demonstrates the potential sexist, discriminatory and gender-based harms that can occur when these technologies are used unchecked.

Purcell’s experience also reflects a disturbing trend in which images, particularly of women and girls, are being sexualised, “deepfaked” and “nudified” without the person’s knowledge or consent.

Read more:
Nine was slammed for ‘AI editing’ a Victorian MP’s dress. How can news media use AI responsibly?

What’s AI got to do with it?

The term AI can include a wide range of computer software and smartphone apps that use some level of automated processing.

While science fiction might lead us to think otherwise, much of the everyday use of AI-assisted tools is relatively simple. We teach a computer program or smartphone application what we want it to do, it learns from the data we feed it, and it applies this learning to perform the task in varying ways.

A problem with AI image editing is that these tools rely on the information our human society has generated. It is no accident that instructing a tool to edit a photograph of a woman might result in it making the subject look younger, slimmer and/or curvier, and even less clothed. A simple internet search for “women” will quickly reveal that these are the qualities our society frequently endorses.

While digitally retouching real photos has been happening for years, fake images of women are on the rise.
Shutterstock

Similar problems have emerged in AI facial recognition tools that have misidentified suspects in criminal investigations due to the racial and gender bias that is built into the software. The ghosts of sexism and racism, it seems, are literally in the machines.

Technology reflects back to us the disrespect, inequality, discrimination and – in the treatment of Purcell – overt sexism that we ourselves have already circulated.

Sexualised deepfakes

While anyone can be a victim of AI-facilitated image-based abuse, or sexualised deepfakes, it is no secret that there are gender inequalities in pornographic imagery found online.

Sensity AI (formerly Deeptrace Labs) has reported on online deepfake videos since December 2018 and consistently found that 90–95% of them are non-consensual pornography. About 90% of them are of women.

Young women, children and teens across the globe are also being subjected to the non-consensual creation and sharing of sexualised and nudified deepfake imagery. Recent reports of faked sexualised images of teenage girls have emerged from a New Jersey high school in the United States and another high school in Almendralejo, Spain. A third instance was reported in a London high school, which contributed to a 14-year-old girl taking her own life.

Read more:
Taylor Swift deepfakes: new technologies have long been weaponised against women. The solution involves us all

Women celebrities are also a popular target of sexualised deepfake imagery. Just last month, sexualised deepfakes of Taylor Swift were openly shared across a range of digital platforms and websites without consent.

While research data on broader victimisation and perpetration rates of this sort of image editing and distribution is sparse, our 2019 survey across the United Kingdom, Australia and New Zealand found 14.1% of respondents aged between 16 and 84 had experienced someone creating, distributing or threatening to distribute a digitally altered image representing them in a sexualised way.

Our research also shone light on the harms of this form of abuse. Victims reported experiencing psychological, social, physical, economic, and existential trauma, similar to harms identified by victims of other forms of sexual violence and image-based abuse.

This year, we have begun a new study to further explore the issue. We’ll look at current victimisation and perpetration rates, the consequences and harms of the non-consensual creation and sharing of sexualised deepfakes across the US, UK and Australia. We want to find out how we can improve responses, interventions and prevention.

How can we end AI-facilitated abuse?

The abuse of Swift in such a public forum has reignited a call for federal laws and platform regulations, moderation and community standards to prevent and block sexualised deepfakes from being shared.

Stunningly, while the non-consensual sharing of sexualised deepfakes is already a crime in most Australian states and territories, the laws relating to their creation are less consistent. And in the US, there is no national law criminalising sexualised deepfakes. Fewer than half of US states have one, and state laws are highly variable in how much they protect and support victims.

But focusing on individual states or countries is not sufficient to tackle this global problem. Sexualised deepfakes and AI-generated content are perpetrated internationally, highlighting the need for collective global action.

There is some hope we can learn to better detect AI-generated content through guidance in spotting fakes. But the reality is that technologies are constantly improving, so our abilities to differentiate the “real” from the digitally “faked” are increasingly limited.

Read more:
Celebrity deepfakes are all over TikTok. Here’s why they’re becoming common – and how you can spot them

The advances in technology are compounded by the increasing availability of “nudify” or “remove clothing” apps on various platforms and app stores, which are commonly advertised online. Such apps further normalise the sexist treatment and objectification of women, with no regard for how the victims themselves may feel about it.

But it would be a mistake to blame technology alone for the harms, or the sexism, disrespect and abuse that flows from it. Technology is morally neutral. It can take neither credit nor blame.

Instead, there is a clear onus on technology developers, digital platforms and websites to be more proactive by building in safety by design. In other words, putting user safety and rights front and centre in the design and development of online products and services. Platforms, apps and websites also need to be made responsible for proactively preventing, disrupting and removing non-consensual content and technologies that can make such content.

Australia is leading the way with this through the Office of the eSafety Commissioner, including national laws that hold digital platforms to account.

But further global action and collaboration is needed if we truly want to address and prevent the harms of non-consensual sexualised deepfakes.

Anastasia Powell receives funding from the Australian Research Council and Australia's National Research Organisation for Women's Safety (ANROWS). Anastasia is also a director of Our Watch (Australia's national organisation for the prevention of violence against women), and a member of the National Women's Safety Alliance (NWSA).

Adrian James Scott receives funding from the Australian Research Council and the Criminology Research Council.

Asher Flynn receives funding from the Australian Research Council, the Criminology Research Council and Australia's National Research Organisation for Women's Safety (ANROWS).

Asia A. Eaton receives funding from the National Science Foundation, Meta, SPSSI, and the Australian Research Council. She is Head of Research for Cyber Civil Rights Initiative.