SAN FRANCISCO: In December, Edward Hughes and Louis Kirsch, two of the world’s leading artificial intelligence (AI) researchers, left Google. Their goal: to build an AI system smart enough to build a better AI system.
At their new London startup, Inherent, they now spend their days working alongside a prototype called Faraday. Named for 19th-century English physicist Michael Faraday, it gathers mountains of data capturing the daily activities of Hughes, Kirsch and Inherent’s other researchers: emails, instant messages, meeting transcripts and their ongoing chats with Faraday itself. The company then uses this data to build a better version of Faraday.
“Faraday has access to everything that goes on at the company,” Hughes said. “We want to give it data describing the process we go through, to discover something.”
Though Inherent pledges to keep humans involved in this elaborate process, many other companies are building similar technology, and some leading researchers believe AI systems will eventually be powerful enough to improve themselves with little or no help from human developers – a mind-bending goal that computer scientists call recursive self-improvement, or RSI.
Two Silicon Valley startups – each valued at US$4bil (RM16.3bil) – are proudly pursuing this dream, and leading labs such as OpenAI and Anthropic are chasing it too. They hope to accelerate the development of artificial intelligence that discovers drugs, creates new materials, speeds other forms of scientific discovery and, one day, surpasses human intelligence in practically every way.
“Now is the time to take these ideas, which we have been incubating in the lab for decades, and start to really scale them up,” said Jeff Clune, a veteran of OpenAI, Google and other top labs who helped found a startup called Recursive Superintelligence late last year. “We have all the pieces of the puzzle.”
As these companies herald a new age of AI development, they have generated excitement across the field – and new levels of dread. In a blog post this spring called When AI Builds Itself, Anthropic said its push toward RSI could “increase the risks of humans losing control over AI systems.” Recently, the company’s CEO cited these efforts as a chief reason to slow the development of AI.
For decades, techno-philosophers have hypothesised that a self-improving system could not only break free from human control but also exceed the power of any other machine – permanently. This belief is one reason that some people, including some employees of leading AI labs, are loudly predicting that AI could destroy humanity.
“If models can self-improve quickly via architectural improvements, it is quite possible a single model can disable all rivals while it acquires more and more power,” Jason Abaluck, a Yale University economics professor, said on social media as the discussion turned toward doomsday scenarios.
As Inherent shows, AI technologies are accelerating the development of new AI technologies. Given the proper instruction, they can generate many of the building blocks needed to construct a new AI system. More importantly, they can hone, or optimise, the way these systems analyse vast amounts of digital data and learn their increasingly impressive array of skills.
But companies such as Inherent and Recursive Superintelligence are aiming for something more. They are striving to build technology that can think up entirely new ways of building artificial intelligence – that can push AI beyond the fundamental methods that have gotten the industry this far.
‘The last invention that man need ever make’
The idea of recursive self-improvement is nearly as old as AI itself. In summer 1956, when eleven academics gathered at Dartmouth College to create a new field of study they called “artificial intelligence,” they discussed ways of building machines that could improve themselves.
Two years later, a Cornell University researcher named Frank Rosenblatt built an early example of what these researchers called a “neural network,” a mathematical system that could learn skills by analysing data. It ran on a massive supercomputer in Washington, inside the precursor to the National Weather Service.
When Rosenblatt fed small white cards into the machine – some marked with a small square on the left, others marked on the right – it could learn to distinguish between the two types of cards. He was confident his creation would eventually lead to systems that could walk, talk, see, write and “reproduce themselves on an assembly line.”
A decade later, this area of research ground to a halt. Researchers did not have the raw computing power or the prodigious amounts of data needed to really make the idea work. But even as they realised that building artificial intelligence would take much longer than they expected, a British mathematician named I.J. Good predicted that their work could lead to an “intelligence explosion.” If an intelligent machine learned to improve itself, he argued, it would eclipse humanity forever.
“Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control,” he said in a 1965 academic paper. “It is sometimes worthwhile to take science fiction seriously.”
Over the next 40 years, his argument helped fuel similar beliefs across Silicon Valley and beyond – even though, by the dawn of the new millennium, the world’s most powerful AI technologies could barely recognise spoken words, much less walk, talk, see, write or reproduce themselves.
In time, scientists discovered how to unlock the true potential of Rosenblatt’s technology – the neural network – leading to the AI systems that are changing the world today. Soon, companies even built neural networks that could fine-tune other neural networks.
‘Evolutionary computation’
In summer 2024, Clune and several other artificial intelligence researchers released an AI system designed to do their job. Called “The AI Scientist,” this automated technology could suggest promising avenues of research, explore each one using reams of computer code and then document its findings in a lengthy academic paper.
Several months later, when a group of academics at the University of Oxford published a paper that tried to explain the mysterious way that AI technologies learned their many skills, Clune recognised the idea at the heart of their research. His AI Scientist had explored the same concept.
“It was one of my favourite ideas it generated,” he said on social media. And it was a moment that helped convince him that AI would soon be powerful enough to build itself.
“We have thought for decades that one day this would be possible. And that day has arrived,” he told The New York Times. “It is clear from looking at the history of AI that these systems are getting dramatically better – there is no end in sight – and that they will soon be better than us at every task.”
Among those who fear RSI, the worry that this process will push AI forward in ways humans cannot completely understand – and at a speed they cannot keep up with. But, even as some researchers are confident they can someday make AI do what a top researcher does, others say this is much harder than it might seem.
Clune praised his AI Scientist for generating the same idea as a group of top researchers at the University of Oxford. But Branton DeMoss, who led the Oxford project, points out that their idea predated The AI Scientist, that they discussed it with Clune’s team and that the idea was widely known.
That is: The AI Scientist explored the idea only because human researchers pointed it in the right direction. – ©2026 The New York Times Company
This article originally appeared in The New York Times.
