ByteDance Ltd is readying an artificial intelligence (AI) model geared for real-time spatial video generation, taking on Meta Platforms Inc and Alphabet Inc in an arena with applications in robotics and autonomous systems.
Founder Zhang Yiming is personally overseeing the development of a new model slated for launch as soon as next month, according to people familiar with the matter. The billionaire has been coordinating the efforts of business units across the company and throwing AI resources and computing capacity behind the effort, the people said, asking not to be identified discussing private information. The timing of the launch is not certain and plans may change, one of the people said.
Built on Seedance – ByteDance’s existing AI model for generating cinematic videos – the model would allow users to create interactive virtual worlds for live streams, short-form dramas and games, the people said. A spokesperson for ByteDance didn’t respond to a request for comment.
Zhang hopes his latest AI model will launch the TikTok creator into the burgeoning arena of world models, armed with top-tier expertise in video. He would be joining the likes of Fei-Fei Li and Yann LeCun in exploring a visual approach to AI that’s considered crucial for robotics, games and self-driving cars.
The move is part of an ongoing strategic shift within ByteDance, as the Beijing-based company redirects much of its business operations toward generative AI. It’s trying to catch up with domestic open-weight rivals like Alibaba Group Holding Ltd and Moonshot AI in shipping frontier language models, while competing against Tencent Holdings Ltd in the enterprise AI software market.
Zhang sees the spatial-video model – which would place the user in a three-dimensional world much like Google’s Genie – as a flywheel, linking ByteDance’s AI models and cloud-computing resources with its content platforms and extended reality arm Pico’s hardware, the people said.
Both ByteDance and Google’s systems are part of a video-centric push to build models that can simulate physical environments for AI agents. Genie was launched earlier this year to let users interact with a real-time rendered world.
If successful, the new model may open a new front in ByteDance’s rivalry with Meta and Apple Inc, which have invested heavily in virtual- and mixed-reality devices. Meta has sought to build a mass-market ecosystem around its Quest headsets, while Apple has positioned Vision Pro as a premium spatial-computing platform. Both have struggled to go mainstream.
ByteDance’s latest model aims to generate virtual worlds that respond to Pico headset users’ voices or movements. The model can offer on-demand videos with a latency of about 0.05 seconds at 20 frames per second and serve spatial-computing environments and interfaces, one of the people said.
The Chinese model seeks to lower the upfront cost of VR adoption by moving the computationally intensive process of generating spatial content to the cloud. That would reduce processing power required in headsets and open a path for cheaper and less sophisticated hardware. Swift content generation might also help shift the battleground for users away from hardware specs to AI models, computing infrastructure and content-distribution platforms.
Seedance has emerged as ByteDance’s signature AI success, even as the company fights a crowded domestic field. It underpins a slew of ByteDance products, including CapCut and Doubao, China’s most popular AI chatbot. The video tool has also become the go-to choice for independent film studios, creators and emerging AI startups.
ByteDance, which Zhang founded alongside Liang Rubo in 2012, has secured a US$30bil (RM121.35bil) loan as it rushes to expand its AI capabilities and amass data centres and other hardware, Bloomberg reported last week. – Bloomberg
