An independent human rights organization says TikTok is hosting and promoting neo-Nazi accounts that are denying the holocaust, glorifying Hitler, supporting white supremacist mass shooters and more. They’re also using generative AI, coded words and audio, and manipulated or obfuscated images to evade filters.
“TikTok hosts hundreds of accounts which openly support Nazism and use the video app to promote their ideology and propaganda,” the Institute for Strategic Dialog report says. “This includes videos featuring Holocaust denial; glorification of Hitler, Nazi-era Germany, and Nazism as a solution to contemporary issues; support for white supremacist mass shooters, and livestreamed footage or recreations of these massacres.
ISD says they found a network of “hundreds” of accounts after initially seeing just one neo-Nazi. And, similarly to how YouTube has been shown to be used by racists to amplify their messages, TikTok’s algorithm works to give people more of what they seem to like. That’s a self-reinforcing loop which leads to more and more of the same kind of content, and when it comes to hate speech or misinformation, the algorithm works exactly the same way on TikTok’s “For You Page.”
“After viewing the pages of 10 accounts in the network and 10 videos total, a new dummy account scrolled through just three videos in TikTok’s For You Page (FYP) before receiving a recommendation for Nazi propaganda,” the ISD report says. “For a longer-standing dummy account which has engaged with more such content, most videos in TikTok’s FYP promote Nazism or hate speech.”
When reported, TikTok is not taking the videos down, or at least not taking them down promptly, ISD says, despite TikTok’s community guidelines around hateful ideologies, promotion of violence, Holocaust denial, and more.
Specifically, TikTok community guidelines state that “we do not allow any hate speech, hateful behavior, or promotion of hateful ideologies.”
A day after reporting, the accounts were still live. 50 of the reported accounts had more than 6.2 million views collectively. Two weeks late, 15 of the accounts were banned, and within a month, 23 of the 50 had been deleted. However, the others will still active, and the ones that were banned had reached over two million views.
The researchers speculate that TikTok eventually removes most offending content, but only after they achieve significant numbers of views.
Tactics used by the neo-Nazi accounts include generative AI to create fear-inspiring images of cultural or racial invasion, use of well-known songs or images commonly used by Nazi or neo-Nazi influencers, and coded language using emojis, acronyms, or numbers to avoid content filters.
(I’m not sharing those images or codes here; you can use the link above to view them on the ISD report page if you choose.)
Some of the content is straight from the 1930s and 1940s in Nazi Germany:
“While most of the early content on the FYP consisted of standard viral TikTok videos, mostly humorous in nature, it took less than 25 videos for an AI-translated speech from Hitler to appear, which was overlaid onto a recruitment poster for a white nationalist group,” the ISD says.
I’ve asked TikTok for a comment on this report, and will update this post as the company responds.
