South Korea is beginning to measure the scale of the mass distribution of pornographic content generated by artificial intelligence (AI) on the Telegram messaging service, after the opening of a police investigation in early September, but victims often feel abandoned.
In most cases, the same pattern repeats itself, according to activist Bang Seo-yoon, who began documenting victims’ testimonies.
Middle and high school students are using photos taken from Instagram accounts to generate pornographic content using AI, which they then share through chat rooms, with the aim of humiliating classmates and teachers.
According to the 18-year-old activist, “it is not only the harm caused by the ‘deepfake’ itself, but the dissemination of these videos among acquaintances that is even more humiliating and painful,” she told AFP.
According to cybersecurity startup Security Hero, the number of these AI-powered schemes has increased by 500% in 2023 and 99% of the victims are women, often famous singers and actresses.
This year, major K-pop labels JYP Entertainment and ADOR, which manage groups such as Twice and NewsJeans, announced plans to take legal action to protect their artists.
But few cases come to fruition: between 2021 and July 2024, 793 crimes related to “deepfakes” were reported, but only 16 people were arrested and prosecuted, according to police data.
However, complaints have multiplied since the scandal broke, with 118 filed in just five days in late August and seven people arrested in a police operation. Only one of them was an adult.
– “Insignificant” sanctions –
Prosecutions are complicated by the fact that South Korean courts rarely issue arrest warrants for minors.
And “the sentences handed down are often insignificant, such as fines or probation, which are disproportionate to the seriousness of the crimes,” said philosophy professor and feminist activist Yoon Kim Ji-young, interviewed by AFP.
She said President Yoon Suk Yeol’s disdain for feminism, which he blames for South Korea’s low birth rate, suggests to men that it is “acceptable to be hostile or discriminatory toward women.”
Victims find themselves “sexually abused and mocked by their classmates in online spaces,” said Kang Myeong-suk, head of victim support at the Women’s Human Rights Institute of Korea (WHRIK).
“The perpetrators of these acts often suffer no consequences” while the victims live “in fear of seeing their manipulated images disseminated by those around them.”
Some people comment online that victims should “get over it” because these doctored images aren’t even real, Kang said.
“But just because the manipulated images aren’t real doesn’t make the pain the victims endure any less authentic.”
– Rejection of fault –
South Korean police are blaming Telegram, which has a reputation for being reluctant to cooperate with authorities. Its founder Pavel Durov was arrested and charged in late August in France for posting illegal content on the app.
Encrypted messaging had already found itself at the heart of a previous scandal of violence and sexual exploitation in South Korea: the “Nth Room” affair, in 2020.
Telegram chat rooms allowed paying members to view photos and videos of violent sexual acts obtained through blackmail and involving minors.
The leader of this network was imprisoned.
The victim of a “deepfake” spread in 2021, who requested anonymity, told AFP that the police’s excuse that the low prosecution rate is due to Telegram’s lack of cooperation is not valid, as many victims manage to identify their perpetrators themselves.
Hers was a classmate from Seoul National University whom she had rarely seen but always found “nice.”
“It was hard to accept,” she said, adding that the police asked her to gather all the evidence herself and then she had to lobby for a trial, which is currently underway.
cdl-hs/ceb/vgu/lgo/tmt
