AI needed to solve AI-driven woes

KUALA LUMPUR: Learning how to adapt to and incorporate artificial intelligence (AI) into data centre systems would be key to Malaysia meeting its aspiration of becoming a regional data centre hub, says an industry player.

Leading critical digital infrastructure provider Vertiv is among the companies that are hoping to help in this regard by highlighting the impact AI can have on sustainable and responsible data centre operations.

Vertiv Malaysia sales director Teoh Wooi Keat (pic) described AI as the “reinvention” of the Internet.

“It will be akin to the ewallet revolution, which has taken the world by storm after it saw gradual widespread introduction across all sectors following the (Covid-19) pandemic,” he said during a presentation he made on AI’s impact on responsible data centre operations on Wednesday.

Teoh said to keep up with this revolution, more efficient and accelerated IT architectures are needed for data centres to meet the requirements of AI workloads.

“AI workloads will require complex calculations with billions of parameters leading to the need for more high-bandwidth memory (HBM) of up to 800Gbps (gigabits per second).

“This is significantly higher compared with the conventional and currently widely-used Double Data Rate 5 Synchronous Dynamic Random-Access Memory (DDR5 SDRAM) with max speeds of 50Gbps that we have now.

“Storage capacities will also need to be improved with current data created yearly already exceeding the 100-zettabyte mark and generative AI expected to further accelerate this trend.

“Networking must also be accelerated as Oracle Cloud Infrastructure (OCI) Networking and AI accelerated instances run on network speeds of 1,600Gbps compared with the traditional instances of 1-10Gbps,” he added.

A zettabyte is one trillion gigabytes.

Teoh said with this sudden jump in requirements, AI was bringing in considerable uncertainty to once-proven infrastructure design practices for data centres.

“Most database racks currently run loads of up to 20 kilowatts (kW) but 80% of IT managers have agreed that rack densities are going up as AI adoption changes the amount of computing power needed per rack.

“In two years, racks are expected to run loads of 40kW and four years later, it may be common for all racks to run loads of up to 80kW, which increases energy usage and heat generated.

“This sharp spike means simply beefing up existing cooling and energy supply solutions to meet the demands of AI workloads at data centres may no longer make economic sense.

“Instead, accelerated AI thermal management solutions – which use precision air-conditioning combined with direct-to-chip and immersion cooling – will be the way forward to cool AI databases both effectively and efficiently,” he added.

To this end, Teoh also presented what stakeholders need to consider when designing future AI-based data centres in the country.

First is to design AI-based data centres’ power and cooling as a single system, with system-level controls to optimise performance and avoid inefficient use or underutilisation of resources in both areas.

“By doing this, these infrastructures should also be designed to be able to handle highly dynamic workloads that are expected of AI-based systems, which can surge by as much as 150%.

“These systems should also be designed for modularity and scalability that enable accelerated future design deployment, making them future-ready to support potential growth of up to 500kW per database rack,” he added.

With this in mind, Teoh said Vertiv is working closely with technology giants Nvidia and Intel to develop power and cooling infrastructure to support their latest, highest-grade AI solutions.

“With Nvidia, the team came up with a novel approach combining two data centre liquid cooling technologies into a single system that has never been seen before.

“Entire servers, with their lower power components, were encased in hermetically-sealed containers and immersed in coolant.

“With Intel, its Gaudi3 AI accelerator was designed to enable both liquid-cooled and air-cooled servers, supported by Vertiv-pumped two-phase (P2P) cooling infrastructure, that recycles coolant,” he added.