Time to regulate algorithms, not access

PUBLIC discussions have drawn parallels between the tragic killing of the 16-year-old female student at a school in Petaling Jaya recently and the Netflix drama series Adolescence to highlight gender-based violence among teenagers.

While we will never know what went through the mind of the 14-year-old perpetrator, the link between red-pill radicalising online content, teenage violence and harm is increasingly being discussed in the public domain. It is no surprise that we are also questioning what exactly we should do about social media.

Now, Communications Minister Fahmi Fadzil is proposing a eKYC (electronic Know Your Customer) process to keep children below 16 off social media platforms. Maybe this will temporarily slow down their access, but I’m sure most will find a way to work around the problem.

More importantly, it doesn’t address the root cause of the problem behind our content landscape – the platform design and algorithmic logic that incentivise and amplify harm.

The effects of social media we have witnessed over the past decade - acceleration of racial polarisation, gender-based violence, child sexual harm - are all symptoms of the social media profit model.

Sure, not all effects are due to social media alone; for instance, gender-based violence has persisted long before the advent of social media. However, the “rabbit-hole” effect of our feeds that dish up content affirming and amplifying hate and anger - the worst biases of humans - is a relatively new threat.

It is no secret that social media algorithms are trained, based on our personal data, to show us exactly what we want to see. Multiple studies have shown that hateful and negative content perform far better than positive and neutral content. A social media user is more likely to watch and share misinformation or polarising content if it confirms their existing beliefs.

Consequentially, engagement-optimised algorithms unintentionally prioritise content that triggers emotional arousal, including hate, anger and extremism.

Here, it is easy to see two effects. One is the vulnerability of users to fall into toxic pockets of social media, like how angry and confused teenage boys become returning spectators of the “manosphere”.

The second is that content producers and influencers need to produce increasingly extreme and sensational content to get views and followers, as they are subjected to the mercy of the algorithm.

If social media content is like a river, then moderating content - and anything that tries to prohibit access - is like trying to stop a gushing flood using a few sandbags. Most harmful content sits in the grey area that escapes moderation. They often don’t meet the threshold to be deleted off the platform, and we run into some serious freedom of speech dilemmas when deciding what deserves to stay up.

Instead, we must look upstream to how content is produced and distributed. The appetite to regulate the actual social media product is muted because it directly speaks to the profit of the platforms. The more time we spend on platforms, the more time we spend watching ads, and the more money they make!

Drawing from new regulation approaches globally, effective regulation must shift from simply moderating content and access to redesigning platform incentives. Rather than asking who sees what, we must ask what objective functions algorithms optimise.

In the United States, legislative bodies have started moving to regulate the design and algorithms of social media, for instance, limiting how social media platforms utilise people's data to feed their content recommendation system, replacing the weight of engagement-based optimisation with user-quality metrics, and placing rate limits for how quickly content can go viral. The default feed for minors should exclude features for revenue maximisation.

Of course, it will not be easy to get compliance, especially with the threat of higher tariffs over

perceived interference with foreign corporate interests. The best metaphor I’ve heard to imagine

this dynamic is that we’re in a modern Opium War: our people are addicted to our feeds (I myself spend two to three hours a day on TikTok), and we have little leverage to intervene in the

product’s entry to our markets.

In other countries, these fights have been taken to the courts, most famously in Brazil where X (formerly Twitter) was temporarily shut down due to the platform reinstating extremist accounts associated with the 2023 Brasília attacks.

I believe social media can be a good and positive space. While we may never be able to go back to the era where we were most excited to use the latest filter on Instagram, we cannot let

our digital public squares (the spaces that shape our democracy, culture and even our capacity for empathy) to be primarily determined by profit motives.

Foreign platforms hold a near-monopoly over a public good. Reclaiming our digital sovereignty begins with demanding transparency, accountability and value-aligned regulation of algorithms that determine what we see, how we feel and, ultimately, who we become online.

JASON WEE

Executive director

Architects of Diversity

Kuala Lumpur