TOKYO: Rakuten Group Inc is expanding its artificial intelligence (AI) team under the stewardship of a Google veteran and building models with a focus on cost efficiency.
Ting Cai, now three years into his tenure at the head of the eCommerce pioneer’s AI team, has the task of creating AI systems that would augment the company’s many businesses and support the handling of commercial transactions at a minimal cost.
He oversees a team that’s grown to 1,000 this year and has a battery of “thousands” of Nvidia Corp chips to work with.
Tokyo-based Rakuten is wrestling with a struggling mobile business and constant competition in online shopping, both of which could get a significant boost from effective deployment of new AI tools.
But the emphasis on doing that profitably from early on is where the company stands out from other large tech firms.
“Rakuten is very business-focused, applying the latest tech to solve customer problems,” Cai, 53, said in an interview.
“In order to do this at a large scale, we have to deliver the maximum margin.
“That’s why reducing the cost is super important for us in deploying generative AI.”
Cai’s team last week unveiled version 3 of the company’s large language model (LLM), which it says is 90% cheaper to run than existing comparable LLMs.
The company segments tasks down to simpler jobs and develops smaller models catered to addressing specific needs for each of its services.
Version 3 activates about 40 billion parameters for each individual token within its 700 billion parameters, while the remaining parameters stay inactive for increased efficiency.
AI functions contributed 10.5 billion yen (US$67mil) to operating income in 2024, and the company aims to double that figure this year.
Intelligent ad targeting and placement have boosted the return on investment for sellers using Rakuten’s online storefront, while AI-powered semantic search and personalised recommendations have lifted user engagement and click-through rates.
“We have observed that users come back more often after using the Rakuten AI Ichiba,” Cai said, referring to the full name of Rakuten’s virtual shopping mall.
“So what we need to do is further reduce the cost of these conversations.
“We want each purchase through conversation to be profitable.”
Before Rakuten, Cai worked on Google Maps and local search services at Alphabet Inc, following a long stint as a software engineer at Microsoft Corp.
He wasn’t too familiar with Rakuten when an executive at the Japanese company contacted him about a job.
It was a “strange company” to Cai at first, he said, but after learning about founder and chief executive officer Hiroshi Mikitani and holding a few conversations with the man himself, Cai realised the two had a lot in common.
The job has turned out to be much more than what Cai believed he was signing up for, he said.
His role has grown into a pivotal one for a company committed to integrating AI across its broad range of services, spanning everything from a mobile assistant to autonomous delivery vehicles.
He is proud of the work his team has accomplished building from the ground up, and Rakuten’s long-term vision includes having business customers tapping its AI tools and expertise. — Bloomberg
