Basis for AI governance

I READ with keen interest a recent article in Futurism.com titled “When AI Is Trained on AI-Generated Data, Strange Things Start to Happen”.

The author was trying to put into context an inherent danger that lurks within the world of generative artificial intelligence if it is not monitored.

This is especially so after the explosive popularity of generative AI following the introduction of ChatGPT in November last year.

Model collapse – a term gaining prominence within the AI domain – signifies a critical hurdle that AI models can encounter during their learning phase. To put it simply, model collapse refers to a scenario where an AI model fails to learn effectively from the data it is provided.

In the context of generative AI, this issue manifests as the model primarily learning from other generative AI systems, often relying on synthetic or artificially generated data.

This concern has not gone unnoticed; AI researchers have been sounding the alarm over the potential risks associated with widespread adoption of generative AI models.

The challenges are multifaceted and requires comprehensive addressing. This entails establishing robust AI governance mechanisms that facilitate collaboration and coordination among key players involved in the development of generative AI models, both locally and globally.

An essential aspect of this collaboration involves information sharing, and collectively ascertaining the origins of the data that AI models are trained on.

Apart from maintaining transparency and accountability in AI development, these efforts are necessary to prevent the proliferation of noisy synthetic data – a phenomenon where erroneous or misleading data generated by AI systems comes into contact with mainstream AI models.

In a more accessible context, envision this as an intricate ecosystem where multiple parties contribute to the growth of AI technology while ensuring it does not deviate from its intended purpose due to inadequate quality control.

For this ecosystem to thrive and produce reliable outcomes, it is essential for these contributors to collectively manage the quality of data and models.

Considering the expanding role of AI in critical sectors such as healthcare, it becomes even more crucial.

AI models might be used in patient risk assessment, for example, where it is used to predict patient outcomes and assess risks based on medical data.

If a predictive model suffers from model collapse, it might become excessively reliant on certain patient profiles or data patterns, leading to biased or inaccurate risk assessments.

In another example, generative AI models can also be used to generate personalised treatment plans based on a patient’s medical history. In the case of model collapse, the AI might generate treatment plans that are very similar to a few standard protocols, disregarding patient individuality. This could lead to sub-optimal, or wasteful (and even dangerous) treatment recommendations.

The call for effective AI governance stems from the need to build a harmonious relationship between technological advancement and responsible usage.

The objective is to strike the right balance that harnesses the potential of generative AI models without succumbing to the risks associated with unchecked development, such as model collapse.

ASSOC PROF TS. DR AZNUL QALID MD SABRI

Faculty of Computer Science and Information Technology,

Universiti Malaya