INVESTORS who have piled into SK Hynix Inc’s US$28bil blockbuster Nasdaq debut should be aware: The business model on which the world’s leading memory chip makers are thriving is set to shift to one that requires a bit more strategic and financial gambling.
So far, SK Hynix Inc is making all the right moves, but it’s unclear how well the field’s leading players will deal with that disruption.
The trigger is of course the artificial intelligence (AI) boom. More powerful AI systems have pushed memory chips to their limits in their job of moving data to and from the processor, the brain of a computer or server.
For most of computing history, calculations on the processor chip were the slow part.
Now things have flipped, since today’s graphics processing units made by Nvidia Corp can do the math far more quickly, while the data being delivered back and forth from the memory chip is still slow in comparison.
Imagine a genius savant who simply can’t write things down fast enough, with their notepad being the memory chip.
“AI machines are starved for memory bandwidth because these models are so big,” former Intel Corp boss Pat Gelsinger said.
That means AI chips aren’t achieving their full potential, and it also limits how much even the most sophisticated chatbots can remember.
Anthropic’s Fable 5, for instance, can only hold about seven novels’ worth of text in its working memory.
Ask it to consider more and it must start dropping what came earlier.
AI builders have been trying for years to get their models to remember far greater amounts, since that would let them carry out more sophisticated, continuous tasks in areas such as finance or back-office corporate work.
But they are constrained in part by physics. Servers, the computers in data centres that run AI systems, use a technique called high-bandwidth memory (HBM).
The idea is to stack fundamental components of a memory chip on top of each other, shortening the distance that data has to travel to and from the processor (the GPU). The technique has flaws, though.
“HBM, as we know and love it today, is a lousy memory. Heat gets trapped in the lower layers of the memory chip, and the rising temperatures force the GPU to suppress its speed,” he said.
One answer is for the world’s three big memory chip producers, SK Hynix, Samsung Electronics Co and Micron Technology Inc, to build memory directly onto the processor itself.
Instead of the genius savant having to write anything down, the notepad is effectively in her head.
That would make AI models many times more powerful, but it also means the three memory chip makers – whose aggregate market value has increased by US$3 trillion in the past year thanks to rampant demand – must change the way they do business.
An industry that has always made standardised chips they could sell like a commodity must now make custom products.
“It’s an entirely new business model for them,” says Gelsinger, who as a general partner at venture capital firm Playground Global is investing in startups that focus on merging memory and computing power into the same piece of silicon.
He expects at least two to three more good years for the big three before the disruption begins to bite, since designing and mass producing new custom chips will take years.
Nvidia has responded to the upcoming changes by striking a US$20bil licensing deal with California-based Groq Inc which builds fast memory onto a processor.
That poses something of a threat to SK Hynix and its peers, who meanwhile will have to get into the business of predicting the future sales of Nvidia, Intel, Alphabet Inc’s Google and other companies that make advanced processors, spending the right amount of money on research and development and chip fabrication to satisfy their demands.
In other words, it’s no longer a matter of building as many of the same memory chips as possible, but making the right bets on who’s winning and losing in the market of selling the brain of a computer.
Choose correctly and you get rewarded with a customer who could be locked in for years; choose wrong and you’re left with a mountain of expensive chips.
SK Hynix chose to hedge its bets in 2024 by partnering with Taiwan Semiconductor Manufacturing Co, which builds chips for almost every major tech company.
That will allow the South Korean memory firm to sell to multiple competing tech giants rather than wagering on just one.
For now, it is riding high on market enthusiasm.
So powerful has the memory boom been that it’s enticed investors to ditch shares in Nvidia in favour of SK Hynix and its peers, as the AI trade shifts to other areas of the infrastructure buildout. (SK Hynix’s shares are up 600% in the past 12 months.)
That’s a startling shift for a commoditised market that spent the previous five decades looking rather dull. “Memory has never been this exciting in history,” Gelsinger effuses.
For the companies and investors piling in, exciting may just be another word for risky. — Bloomberg
Parmy Olson is a Bloomberg Opinion columnist covering technology. The views expressed here are the writer’s own.
