"A Computer Just Passed the Turing Test in Landmark Trial" — Or Did It? The Hype on That One Lasted About a Day

David Klinghoffer


The hype on this one, with its promised overthrow of the human’s mind exceptional status, lasted about a day. Here’s how science reporting works. The press release came out on June 8 from the University of Reading:

An historic milestone in artificial intelligence set by Alan Turing — the father of modern computer science — has been achieved at an event organised by the University of Reading.

The 65 year-old iconic Turing Test was passed for the very first time by computer programme Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

‘Eugene’ simulates a 13-year-old boy and was developed in Saint Petersburg, Russia. The development team includes Eugene’s creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

The dutiful parroting commenced June 9.

"A computer just passed the Turing Test in landmark trial" — headline, Washington Post

"The age of robots is here" — CNN

As of today, June 10, it’s all over, with the widespread realization that the story points out instead how far computers are from being able to simulate human intelligence.

See the takedown in The New Yorker by MIT cognitive scientist Gary Marcus, who points out that if anything "Eugene Goostman" is a reminder that we’ve pretty much given up on a computer meaningfully being able to do the things that a clever child does routinely.

Goostman is little more than an ingenious "parlor trick," an exercise in "bluffing," with roots traceable to the program "ELIZA" back in the Sixties:

Goostman, like ELIZA, relies mainly upon pattern recognition, not genuine understanding. It is a refinement of an old idea, not a fundamental change in artificial intelligence.

Almost nobody in A.I. these days seems to aim for what Alan Turing himself envisioned — a flexible, general-purpose intelligence of the sort that human beings have, which allows any ordinary individual to master a vast range of tasks, from tying his shoes to holding conversations and mastering tenth-grade biology. In the years since Turing, many machines have mastered individual tasks, like playing chess (IBM’s Deep Blue exceeded even the best humans) and even Jeopardy (IBM’s Watson). But each such program is tailored to a particular task, and none has possessed the sort of broad intelligence that characterizes humans. No existing combination of hardware and software can learn completely new things at will the way a clever child can.

As I have learned from two decades of work as a cognitive scientist, the real value of the Turing Test comes from the sense of competition it sparks amongst programmers and engineers. So, in the hope of channelling that energy towards a project that might bringing us closer to true machine intelligence — and in an effort to update a sixty-four-year-old test for the modern era — allow me to propose a Turing Test for the twenty-first century: build a computer program that can watch any arbitrary TV program or YouTube video and answer questions about its content — Why did Russia invade Crimea?" or "Why did Walter White consider taking a hit out on Jessie?" Chatterbots like Goostman can hold a short conversation about TV, but only by bluffing. (When asked what "Cheers" was about, it responded, "How should I know, I haven’t watched the show.") But no existing program — not Watson, not Goostman, not Siri — can currently come close to doing what any bright, real teenager can do: watch an episode of "The Simpsons," and tell us when to laugh.

�A triumph for Artificial Intelligence? Not so much. When it comes to intelligence, what Wesley Smith calls "human exceptionalism" remains the best science that we have.

I’m on Twitter. Follow me @d_klinghoffer.

Image: Statue of Turing by Stephen Kettle/Wikipedia.