Sometimes the unasked question reveals most. In exploring AI systems, and whether do or ever will achieve human-like intelligence, the unasked question is “What does it mean to be intelligent?” The answer is elusive since we have a sample set of precisely one species: humans. (Other species are certainly intelligent, but AI developers seek human-like intelligence so that humans may use the systems.) And, while we agree that humans are intelligent, we cannot describe what we mean, measure what we mean, or agree on what we mean.

Alan Turing, a member of England’s World War II “Code Breakers,” established the foundation for all modern computing, a general purpose computing engine design that now bears his name: the Turing Machine. Every computer we use, despite their outward differences — with some shaped like phones and others filling large rooms — follow his principles. Turing also, long before anyone coined the term artificial intelligence, considered the possible rise of intelligent machines. He even devised a test of machine intelligence that also bears his name: the Turing Test. His test is not hard to state or grasp. If a reasonably intelligent adult can hold a conversation with an unseen interlocutor (Turing envisioned using a teletype whose cables led into another room) and cannot tell if the hidden entity is human or hardware, then the machine passes the test; it has displayed human intelligence.

The Turing Test is elegant and simple. And, for those reasons, like intelligence, it is hard to pin down. Would, say, Microsoft’s Tay — their social Twitter bot that, in less than 24 hours, had to be shut down for spewing sexual and racist comments — pass? A popular 1970s program, Eliza, mimicked a Rogerian psychologist by picking up bits and pieces of the patient’s statements to fill and create subsequent questions. Eliza fooled more than few. (Try it here and see what you think.) More recently, and more seriously, Russian researchers created an artificial intelligence that mimicked a teenage boy. Some felt it passed the Turing Test. Others, however, used the same occasion to point out the test’s flaws and unstated assumptions. More than one science fiction movie has run with Turing’s ideas as its premise.

Despite the headlines, no machine has yet come close to passing a serious Turing Test. A large portion of web security relies on this fact. Anyone using the Internet has likely run into a reverse Turing Test: CAPTCHA images. These images, rather than test if a computer has achieved human intelligence, count on it that they have not. Passing the test, by interpreting the image correctly, proves you are human and not a machine.

Now, Dr. Erik Larson, an AI research scientist and entrepreneur has published a deep and thoughtful analysis of IBM’s Jeopardy!-winning machine, Watson. Dr. Larson, who has written previously on the overselling of AI’s accomplishments, here at Evolution News and elsewhere, asks: Since Jeopardy! is a game that requires, at an advanced level, understanding questions and choosing answers, does winning constitute passing the Turing Test? IBM’s scientists never claimed that Watson did pass such a test, but, by examining how Watson won, Larson uncovers why no machine does pass and why it’s unlikely they ever will.

Larson takes an extensive gander at how Watson works. It’s hard to read the details and not be amazed at the engineering prowess of IBM’s scientists. Their results with Watson, while building on and re-using earlier work, moved the field of question-and-answer systems significantly ahead. Watson won fairly and squarely. It received the question (albeit in an electronic form) at the same time as the other contestants. It had to “decide” if it should answer or not (wrong answers would cost money and standing). It used an electronic “thumb” to hit the buzzer before responding. It had to place bets for Daily Doubles and in the Final Jeopardy round. It could not search the Internet (though, arguably, Watson had stored in advance a good deal of its most popular content). That is, it had to play the game Jeopardy!, as Larson says, with “no tricks, no gimmicks.”

Watson, as Larson explains, is not one large, monolithic system. It is a coordinated collection of thousands of smaller programs, an army of software minions designed to achieve specific purposes. IBM created logic and learning systems to distill the minion’s results into an answer and confidence quotient. They then tweaked the algorithms through training with millions of documents and thousands of questions from prior Jeopardy! games. Then Watson played against itself and others, over, and over, and over again. Each time, IBM’s scientists examined the results to improve Watson’s performance. They even included a few tricks, such as favoring Wikipedia article titles once they learned that, from them, they could answer 95 percent of the Jeopardy! questions. IBM completed a difficult and years-long feat of engineering with Watson.

Some readers may find the detail of Dr. Larson’s article overwhelming. But no one who reads his analysis can walk away without feeling immense awe at what IBM accomplished. What started as a bet and nearly a joke became an outstanding milestone in AI research and a whole new business for IBM.

But Dr. Larson’s analysis also shows something else, which I have raised before. Computers are machines into which we deposit slices of our own minds, recoded and reworked into a form the machine can use. It’s like carefully teasing out and apart something we know, something useful to others, restating it in the restricted form Turing Machines require, and distributing the encoded intelligence for use. These “mind slices” embody the real intelligence within these machines. It’s not that AI machines are developing intelligence so much as that we, as intelligent agents, have found a clever, if complex and difficult, way to capture and reuse portions of that elusive thing we call intelligence.

IBM is promoting Watson to assist doctors and nurses in their practices. To do so, as Dr. Larson points out, requires IBM to feed Watson hundreds of thousands of new documents and to wholly retrain Watson. The underlying system, the collection of software minions and the logic to usefully collect their results, remains much the same. But the long hours spent tuning, adjusting, and preparing Watson to win at Jeopardy! are gone, if, for no other reason, doctors, nurses, and the rest of us start with questions for which we want answers, not answers in search of a question.

For all the hoopla surrounding each amazing AI advance, from IBM’s Watson to Google’s more recent Go-conquering machine, AlphaGo, we forget one critical detail in our amazement: Each of these machines does just one thing. They may do it remarkably well and fast, but that is all they can do. Watson cannot dance, clap, or take a bow. It cannot write a book, play the piano, or sing a song. It cannot drive a car, mow the lawn, or weed the garden. It cannot tell jokes and it does not laugh. It cannot recognize pictures of cats or identify faces. It cannot play Go or Chess. IBM is hoping it can assist in answering medical questions. We know it can win at Jeopardy! Watson does what all AI systems do: It captures and replays just one human ability.

When that captured, such as the skill of a Jeopardy! champion, is beyond what other humans can do, than it is useful. Otherwise it is an oddity and a toy. Erik Larson confirms this when he says that “for all its impressive performance on Jeopardy!, the Watson system still tells us little about the bigger questions of AI.“

Photo: Statue of Alan Turing, Bletchley Park, by Ian Petticrew [CC BY-SA 2.0], via Wikimedia Commons.