For the first time, a 'supercomputer' deceived real humans into thinking that it’s a 13-year-old boy, scoring a historic win for artificial intelligence – but then again...
A Russian "supercomputer" has been claimed as the first computer in the world to be mistaken for a real person during a competition slash experiment based on the so-called "Turing Test" conducted at the Royal Society in London to mark the 60th anniversary of the death of Alan Turing, "father of modern computing".
During the test on June 7, five supercomputers were presented with a series of unrestricted questions by a panel of judges via a computer interface for a period of five minutes per chat. The judges then had to decide, based on the replies, whether or not they interacted with a real person or something that's just pretending it's real.
It all seems simple enough. But with 150 separate chats with a panel of 30 judges – not to mention the 25 "hidden humans" thrown into the mix with the five supercomputers, to substantiate the results and no doubt to make the judges work harder – it was a test that would make any robot weep.
The Turing Test – a test to determine whether artificial intelligence has risen to the level of human intelligence – was hypothesised by Alan Turing, a British mathematician considered a forerunner in computer science and who played a key role in breaking Nazi Germany's Enigma code during World War II.
In a journal article entitled "Computing Machinery and Intelligence" published in 1950, Turing speculated on the possibility of computers to 'think' independently. He wrote:
"I believe that in about 50 years' time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70% chance of making the right identification after five minutes of questioning."
Though off by a decade or so, it seems that Turing's prediction may have finally come true.
'I am Eugene'
Of the five "supercomputers" that competed – Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal – it was the Russian computer programme, a simulation of a 13-year-old boy named "Eugene Goostman" originally from Odessa in the Ukraine, who persuaded the judges 33% of the time that it was human. And based on what Turing wrote, apparently 30% is good enough for a pass.
”In the field of artificial intelligence there is no more iconic and controversial milestone than the Turing Test, when a computer convinces a sufficient number of interrogators into believing that it is not a machine but rather is a human,” said Prof Kevin Warwick, a visiting professor at the University of Reading, west of London, who co-organised the competition with his colleague Huma Shah.
While some experts claim that the Turing Test had already been passed, Warwick said the Royal Society experiment went further – including the random nature of the questions – and was independently verified. ”We are therefore proud to declare that Alan Turing’s Test was passed for the first time on June 7,” he proudly announced.
The Russian creator of Eugene, scientist Vladimir Veselov currently based in New Jersey, US, said the result was a remarkable achievement. ”We spent a lot of time developing a character with a believable personality,” he said. ”This year we improved the ‘dialogue controller’ which makes the conversation far more human-like when compared to programmess that just answer questions."
Although Eugene has competed and even won similar Turing Test competitions in the past, this was the first time it passed the "30%" mark. Now that Eugene has "beaten" the Turing Test, Veselov plans to develop it further: "We plan to make Eugene smarter and continue working on improving what we refer to as ‘conversation logic’.”
While Warwick and Veselov are celebrating Eugene's historic win for artificial intelligence, others have expressed doubts, saying that their claim to passing the Turing Test is nothing but a load of bollocks.
Responding to a Guardian UK article about Eugene's victory, Prof Robert Epstein from the American Institute for Behavioural Research and Technology said that Turing never set 30% as a passing mark, claiming that Warwick had interpreted Turing's ideas too literally.
Epstein argued that the scoring method used by the Loebner Prize, an annual Turing Test competition which he co-founded in 1990, is a more exacting standard: "After lengthy conversations with both hidden humans and hidden computers, a panel ranks the humanness of each, and when the median rank of a computer exceeds the median rank of a human, it wins. No computer has ever crossed that line in the more than 20 years the contest has so far been held."
Meanwhile, British tech website The Register dismissed the test as a media publicity stunt by Warwick, whom they've nicknamed "Captain Cyborg" for what they claim as his tendency to blow things out of proportion, as when Warwick claimed in 1991 that a real-life Terminator would be invented within a decade. The website also dismissed Eugene: "Acting like a 13-year-old is a cunning way to disguise its crude intelligence."
Other commentators pointed out the fact that Eugene is not in fact a "supercomputer" as has been claimed in media reports, but merely a chatbot – a computer chat software designed to emulate human-like responses – that lacked the necessary qualities that would pass as human intelligence. Some have even criticised the Turing Test itself, saying it's logically flawed, as no one has yet been able to quantify nor create a definitive test to authenticate human intelligence, much less artificially created ones.
At any rate, readers can decide for themselves whether or not Eugene can pass off for a real boy. An online version of Eugene, complete with the goofy tagline of "the weirdest creature in the world", will apparently answer your questions about anything. But try not to expect anything overly intelligent as transcripts of previous chats tend to swing towards the bratty end of the conversation norms. Below is an example:
USER: What is the square root of a negative number?
EUGENE: Sorry, I can't calculate myself, but my well-educated home-pet, guinea pig, says he got an overflow trying to obtain the result :-(
If Eugene seems less intelligent than some proponents of artificial intelligence would have led you to believe, let's just say that it wasn't designed with Einstein in mind.
The 'real' Turing test
As for Alan Turing, the real test was his brilliant yet tragic life. Trained as a mathematician at Cambridge University and later completing his Phd at Princeton, he is often hailed as a genius who laid the groundwork for modern computing based on his idea of a 'Universal Machine' – a single machine that can conceivably perform any task that requires computing.
During World War II, his mathematical genius came to the fore when he played a key role in helping the British government break the Enigma code that was used by the Nazis in their communications, eventually leading to the Allied Forces' victory in 1945.
The two papers he wrote on the subject Report on the applications of probability to cryptography and Paper on statistics of repetitions were so valued for their contents that the British authorities kept them secret until 2012. For his contributions to the war effort, Turing was awarded an OBE by King George in 1945, even though only a handful of people knew why he received it at the time.
Despite his genius and contributions, Turing's life ended on a tragic note. Two years after being convicted of the then-crime of homosexuality and suffering from depression stemming from physical changes brought on by his sentence of forced injections of female hormones, Turing committed suicide in 1954. He was 41.
In the years since his passing, as the tide of public opinion on homosexuality turned and his ideas on computer science gained wider acceptance, Turing amassed a following of supporters that included academics, tech geeks and human rights activists who launched a campaign to get him an official pardon from the British government.
Their efforts paid off in 2009, when British Prime Minister Gordon Brown made an official public apology on behalf of the British government for "the appalling way he was treated". On Christmas Eve last year, Turing's name was finally in the clear when Queen Elizabeth II granted him a posthumous pardon.
Sources: AFP RelaxNews, New Scientist, The Guardian UK, The Register UK, Wikipedia
Report it to us.
Thank you for your report!