In recent weeks, two related developments have threatened to reshape the fast-moving artificial intelligence boom: Companies using the most advanced AI tools have begun to baulk at the cost, and new open-source models from China have become nearly as powerful as the proprietary technology from large US labs.
For Vipul Ved Prakash, a co-founder and CEO of Together AI, a startup specialising in open-source AI, that confluence creates a window of opportunity.
Company margins “are being eaten” by the costs of using closed models, such as those offered by OpenAI and Anthropic, Prakash said.
Executives have begun to push back against “tokenmaxxing” – tokens being the units of data used by AI models – while searching for solutions to keep expanding their use of the technology. As companies adopt open-source models, which can be used and modified freely, Together AI’s business has surged.
The venture capital community has taken notice.
On July 1, Together AI announced an US$800mil (RM3.26bil) funding round at an US$8.3bil (RM33.88bil) valuation, bringing its total funding to US$1.3bil (RM5.31bil). The new round was led by Prosperity7 Ventures, the venture arm of Saudi Aramco, Saudi Arabia’s state oil company, with participation from Nvidia, Vista Equity Partners, General Catalyst and others.
Prakash, a serial entrepreneur and former Apple executive, founded the San Francisco-based Together AI with four academics in 2022. The company’s chief scientist, Tri Dao, wrote FlashAttention, a seminal algorithm for improving model speed, which is now a core part of Together AI’s product.
Demand for Together AI’s products is soaring. In the past year, the company has had a 10,000-fold increase in the number of tokens it processes per month. Last quarter, the annual rate of its revenue grew to US$1.2bil (RM4.9bil).
Together AI’s growth is a testament to rising demand for open-source models, many of which are made in China. In June, a Chinese startup, Z.ai, released an AI model with abilities similar to Anthropic’s powerful Fable and Mythos models.
Some experts are worried about US companies’ adoption of Chinese open-source models. They cite concerns that the models are built by companies connected to the Chinese government and that developers have illicitly leveraged American technology.
Despite those misgivings, the relatively low price of Chinese models is enticing. They can be one-fiftieth the cost to use compared with their American counterparts on a per-token basis, according to a recent JP Morgan report.
Open-source processing on OpenRouter, an AI model marketplace, rose to 65% in June from 34% in January, according to a Citi analysis that Reuters reported. Microsoft, which helped foster the AI boom with its early investment in OpenAI, is considering hosting a version of DeepSeek, an open-source Chinese model, on its Copilot program.
Together AI helps give companies access to those cheaper open-source models to power their AI products. The company rents and buys AI computing chips and uses specialised software to run AI models faster and more efficiently for customers – a process called “inference.”
The startup then offers hundreds of open-source models on those chips that can be tailored to a company’s needs, removing a firm’s need to hire an engineering team to do that on its own.
Ashwin Sreenivas, a co-founder of AI startup Decagon, said using Together AI had saved his company a “massive” amount. Decagon, which makes an AI customer service product, uses a mix of closed models from leading AI companies including OpenAI as well as open-source models from Together AI. Sreenivas estimated that tasks moved from closed models to Together AI cost one-fifth to one-seventh as much.
The demand for open-source models is surging as Anthropic and OpenAI gear up for major public offerings. “As they are going to the public markets, a lot of other companies are thinking of a future without them,” said Prakash, Together AI’s CEO.
The New York Times reported that OpenAI was considering delaying its offering, in part because of financial challenges.
Together AI plans to spend most of its newly raised capital on research and development. It’s also leveraging some of its funding to build out more compute capacity. To do so, its investors are proving helpful, including by introducing Together AI “to the right people who might be looking at building more capacity,” said Abhishek Shukla, a managing director at Prosperity7.
Competition among companies hosting open-source models is hot. Together AI’s rival Baseten raised US$1.5bil (RM6.12bil) at a US$13bil (RM53.07bil) valuation. Another competitor, Cerebras, held an IPO in May and now has a market value of around US$50bil (RM204.11bil).
Shukla believes that Together AI will try to follow suit. The company, he said, is “headed towards the public markets.” – ©2026 The New York Times Company
This article originally appeared in The New York Times.
