At first glance, graphic videos circulating online in China show some of the country’s most prominent female stars engaged in sexually explicit acts. Then the news dropped: The images are fake.

Illegal peddlers are selling videos that use AI software to stitch faces of celebrities onto pornographic clips, according to The Beijing News and Global Times. These recent reports are highlighting the difficulties of combating the spread of fake videos online.

These types of manipulated videos, dubbed deepfakes, began surfacing in the dark corners of the internet around two years ago. They’ve since been used to make Barack Obama and Mark Zuckerberg say things they never said. And equally disconcerting, we’ve also seen the faces of Scarlett Johansson and Gal Gadot inserted into porn films.

In China, deepfakes were thrown into the national spotlight in February when a video that splices a major actress into a 25-year-old TV show became a top-trending story on Weibo. The clip itself seemed harmless, but it triggered questions on what could happen if the technology was used for more nefarious purposes.

Now, it appears, the country is finally getting some disturbing answers.

A deepfake video showing the face of major Chinese star Yang Mi stitched onto a nude body. (Picture: The Beijing News via Xigua Video)

Beijing News reporters on Thursday said they found listings on Alibaba’s second-hand marketplace Xianyu and Baidu’s forum page Tieba advertising deepfakes of female celebrities. Some offer bundles of dozens to hundreds of clips, sold for anywhere from under US$2 to about US$23. Others say they can face-swap any individual, whether a celebrity or not, into any video.

On Friday, Xianyu appeared to have blocked terms such as “AI face-swap” and “face-swap software.” Searching “face-swap video,” however, still yielded listings purporting to sell deepfake software and do-it-yourself tutorials. We were unable to find any explicit listings on Baidu Tieba, although there are still sporadic posts sharing deepfake videos.

Both platforms told us they have taken down the illegal listings. Baidu said they’ve reported the cases to the police. Xianyu said they are continuing to monitor product listings on the platform.

(Abacus is a unit of the South China Morning Post, which is owned by Alibaba.)

China Internet Report 2019

One of the challenges of fighting deepfake pornography is that it’s so easy to create. Software can be easily obtained on the internet, allowing anyone to generate deceptively real videos without learning AI coding.

Just a few weeks ago, software called DeepNude triggered an outcry. The app, which was later taken down by the creator, let users upload pictures of clothed women and digitally undress them.

The original DeepNude app has been pulled, but copies can still be found online. (Picture: DeepNude)

Pornography is illegal in China, though there are currently no official laws that deal specifically with other types of deepfakes. That could change soon.

Efforts are underway to introduce a law that prohibits the use of technology to create fake, non-consensual video or audio of any person, according to a China Daily report in April.

Parts of the US have similarly begun to erect legislation against certain types of AI-tampered videos. Earlier this month, Virginia became one of the first states in the country to criminalize the spread of sexually explicit deepfake videos without consent.