Stopping online hate speech is hard. New research teaches machines to find white nationalist content

Big tech companies like Facebook and Twitter have been accused of stifling free speech by removing users who violate community guidelines prohibiting hate speech. — Relaxnews

Mitigating the impact of online extremism has proven a complicated task for companies that want to protect free expression.

Social media companies have struggled to moderate hate speech and adapt to changes in how white supremacists spread their views on digital platforms. Libby Hemphill, an associate professor at the University of Michigan School of Information, believes machine learning technology might provide an answer.

“We know that white supremacists and other types of extremists use social media to talk to each other, to recruit, to try to get their message to go mainstream,” Hemphill said. “The challenge has been that the platforms haven’t really stepped up to fight hate on their platforms.”

Through a partnership with the Anti-Defamation League, Hemphill set out to teach algorithms to distinguish white supremacist and extremist speech from the typical conversations people are having on social media. Turns out, extremist groups are pretty good at hiding in plain sight but algorithms can become even better at finding them.

“We can’t do content moderation without some machine assistance,” Hemphill said. “There’s just too much content.”

Hemphill started by collecting a sample of 275,000 blog posts from Stormfront, a white nationalist website. The data was fed to algorithms used to study the sentence structure of posts, detect specific phrases and flag recurring topics.

The goal was to train a machine to identify toxic language using the Stormfront conversations as a model. Algorithms compared the Stormfront data to 755,000 Twitter posts from users affiliated with the alt-right movement and another set of 510,000 Reddit posts collected from general users.

The results are still being compiled. Hemphill is hoping to unveil a public tool later this fall.

Big tech companies like Facebook and Twitter have been accused of stifling free speech by removing users who violate community guidelines prohibiting hate speech. Offending posts are identified by algorithms trained to detect hate speech and through reports from users.

However, advocacy organisations aren’t satisfied with enforcement standards.

The Center to Counter Digital Hate found the top five social media companies took no action on 84% of anti-semitic posts reported to them. CCDH, a non-profit group in the United States and the United Kingdom, flagged 714 posts through the platforms’ user reporting tools. The anti-semitic posts were viewed up to seven million times.

“Platforms must aggressively remedy their moderation systems which have been proven to be insufficient,” CCDH CEO Imran Ahmed wrote in a study announcing the group’s findings.

Steps to ban white nationalist content also inspired the rise of new platforms with more relaxed content standards. Gab, MeWe and Parler purport to champion free speech but have been criticised as havens for extremists.

Gab CEO Andrew Torba promotes his website as “the only place you can criticise groups like the American Jewish Congress and the ADL” in emails to Gab’s user base.

Hemphill said there’s significant conflict revolving around what constitutes hate speech and what social platforms should do about it. She also acknowledged concerns that machines can make mistakes and unfairly punish users.

“The challenge is that we as a community don’t share a set of values,” Hemphill said. “We disagree about what is hateful and what ought to be permissible. One of the things that I would like to see come out of work like this is more explicit discussions about what our values are and what is okay with us and what isn’t. Then we can worry about what we ought to teach machines.”

Finding the line can be tricky. Accurately identifying nuanced racial stereotypes and microaggressions is harder than finding slurs, Hemphill said. There’s also a large number of posts containing meme images and videos that are harder to accurately catalogue.

Stormfront was used as a baseline in Hemphill’s research because the group openly identifies as a forum for white nationalists. Alt-right Twitter users were selected from a study commissioned by the Centre for American Progress.

Beyond the use of racial slurs and other kinds of toxic language, Hemphill said subtle differences were found between white nationalists and the average internet user. For example, white nationalists swear less often, possibly a tactic to appear more palatable to mainstream audiences.

“They are sort of politely hateful,” Hemphill said. “If you’ve spent any time on the Internet at all, you know that it’s a pretty profane place, but white supremacists are not profane. They are marked by what they don’t do as well.”

Hate speech makes up a relatively small amount of content on the Internet, Hemphill said, though it makes a large impact on whoever sees it. She plans to collect more data from smaller websites like Gab, 4chan and Parler to continue teaching algorithms to identify hate speech.

Ultimately, the goal is to encourage social media companies to create inclusive spaces for discussion.

“It’s OK to be freaked out about big tech, and it’s OK to be freaked out about academic researchers and what we may be doing,” Hemphill said. “I think the more this conversation happens in the open and the more data is shared among folks who are trying to understand how these decisions get made, the better, more just decisions about what is permissible will come from that openness.” – News Service

Article type: metered
User Type: anonymous web
User Status:
Campaign ID: 1
Cxense type: free
User access status: 3
Join our Telegram channel to get our Evening Alerts and breaking news highlights

Next In Tech News

Telecom Italia shares slide further as doubts over KKR's bid intensify
Heritage land dispute over Amazon's new Africa HQ goes to court
British digital banking app Revolut launches U.S. stock trading
‘Game changer’ electric moped batteries spread from Taiwan across Asia
S.Korea's LG Energy Solution sees record retail demand for IPO
Five taken to hospital after fire linked to charging ebike battery breaks out in SG flat
Turkey's Trendyol plans dual IPO when revenue abroad is 30-35%
TikTok owner ByteDance reorganises strategic investment team
Chip supplier ASML forecasts strong 2022 despite fire hit
Booming NFT art market plagued by ‘mind-blowing’ fraud

Others Also Read