Intelligence is not Artificial

Why the Singularity is not Coming any Time Soon And Other Meditations on the Post-Human Condition and the Future of Intelligence

by piero scaruffi
Cognitive Science and Artificial Intelligence | Bibliography and book reviews | My book on consciousness | Contact/feedback/email

(These are excerpts from my book "Intelligence is not Artificial")

An Easy Science

When a physicist makes a claim, an entire community of physicists is out there to check that claim. The paper gets published only if it survives peer review, and usually many months after it was written. A discovery is usually accepted only if the experiment can be repeated elsewhere. For example, when OPERA announced particles traveling faster than light, the whole world conspired to disprove them, and eventually it succeeded. It took months of results before CERN accepted that probably (not certainly) the Higgs boson exists.

Artificial Intelligence practitioners, instead, have a much easier life. Whenever they announce a new achievement, it is largely taken at face value by the media and by the A.I. community at large. If a computer scientist announces that her or his program can recognize a cat, the whole world posts enthusiastic headlines even if nobody has actually seen this system in action, and nobody has been able to measure and doublecheck its performance: can i make a video of a cat and feed it into the program and see if it recognizes the cat? When in 2012 Google announced that "Our vehicles have now completed more than 300,000 miles of testing" (a mile being 1.6 kilometers for the rest of the world), the media simply propagated the headline without asking simple questions such as "in how many months?" or "under which conditions?" or "on which roads"? "at what time of the day"? Most people now believe that self-driving cars are feasible even though they have never been in one. Many of the same people probably don't believe all the weird consequences of Relativity and Quantum Mechanics, despite the many experiments that confirmed them.

The 2004 DARPA challenge for driverless cars was staged in the desert between Los Angeles and Las Vegas (i.e. with no traffic). The 2007 DARPA urban challenge took place at the George Air Force Base. Interestingly, a few months later two highly educated friends told me that a DARPA challenge took place in downtown Los Angeles in heavy traffic. That never took place. Too often the belief in the feats of A.I. systems feels like the stories of devout people who saw an apparition of a saint and all the evidence you can get is a blurred photo.

In 2005 the media reported that Hod Lipson at Cornell University had unveiled the first "self-assembling machine" (the same scientist in 2007 also unveiled the first "self-aware" robot), and in 2013 the media reported that the "M-blocks" developed at the MIT by Daniela Rus' team were self-constructing machines. Unfortunately, these reports were wild exaggerations.

In May 1997 the IBM supercomputer "Deep Blue", programmed by Feng-hsiung Hsu (who had started building chess-playing programs in 1985 while at Carnegie Mellon University), beat then chess world champion Garry Kasparov in a widely publicized match. What was less publicized is that the match was hardly fair: Deep Blue had been equipped with an enormous amount of information about Kasparov's chess playing, whereas Kasparov knew absolutely nothing of Deep Blue; and during the match IBM engineers kept tweaking Deep Blue with heuristics about Kasparov's moves. Even less publicized were the rematches, in which the IBM programmers were explicitly forbidden to modify the machine in between games. The new more powerful versions of Deep Blue (renamed Frintz) could beat neither Vladimir Kramnik, the new world chess champion, in 2002 nor Kasparov himself in 2003. Both matches ended in a draw. What is incredible to me is that a machine equipped with virtually an infinite knowledge of the game and of its opponent, and with lightning-speed circuits that can process virtually infinite number of moves in a split second cannot beat a much more rudimentary object such as the human brain equipped with a very limited and unreliable memory: what does it take for a machine to outperform humans despite all the technological advantages it has? Divine intervention? Nonetheless, virtually nobody in the scientific community (let alone in the mainstream media) questioned the claim that a machine had beaten the greatest chess player in the world.

If IBM is correct and, as it claimed at the time, Deep Blue could calculate 200 million positions per second whereas Kasparov's brain could only calculate three per second, who is smarter, the one who can become the world's champion with just three calculations per second or the one who needs 200 million calculations per second? If Deep Blue were conscious, it would be wondering "Wow, how can this human being be so intelligent?"

What Deep Blue certainly achieved was to get better at chess than its creators. But that is true of the medieval clock too, capable of keeping the time in a way that no human brain could, and of many other tools and machines.

Finding the most promising move in a game of chess is a lot easier than predicting the score of a Real Madrid vs Barcelona game, something that neither machines nor humans are even remotely close to achieving. The brute force of the fastest computers is enough to win a chess game, but the brute force of the fastest computers is not enough to get a better soccer prediction than, say, the prediction made by a drunk soccer fan in a pub. Ultimately what we are contemplating when a computer beats a chess master is still what amazed the public of the 1950s: the computer's ability to run many calculations at lightning speed, something that no human being can do.

IBM's Watson of 2013 consumes 85,000 Watts compared with the human brain's 20 Watts. (Again: let both the human and the machine run on 20 Watts and see who wins). For the televised match of 2011 with the human experts, Watson was equipped with 200 million pages of information including the whole of Wikipedia; and, in order to be fast, all that knowledge had to be stored on RAM, not on disk storage. The human experts who competed against Watson did not have access to all that information. Watson was allowed to store 15 petabytes of storage, whereas the humans were not allowed to browse the web or keep a database handy. De facto the human experts were not playing against one machine but against a whole army of machines, enough machines working to master and process all those data. A fairer match would be to pit Watson against thousands of human experts, chosen so as to have the same amount of data. And, again, the questions were conveniently provided to the machine as text files instead of spoken language. If you use the verb "to understand" the way we normally use it, Watson never understood a single question. And those were the easiest possible questions, designed specifically to be brief and unambiguous (unlike the many ambiguities hidden in ordinary human language). Watson didn't even hear the questions (they were written to it), let alone understand what the questioner was asking. Watson was allowed to ring the bell using a lightning-speed electrical signal, whereas the humans had to lift the finger and press the button, an action that is order of magnitudes slower.

Over the decades i have personally witnessed several demos of A.I. systems that required the audience to simply watch and listen: only the creator was allowed to operate the system.

Furthermore, some of the most headline-capturing Artificial Intelligence research is supported by philanthropists at private institutions with little or no oversight by academia.

Many of the A.I. systems of the past have never been used outside the lab that created them. Their use by the industry, in particular, has been virtually nil.

For example, on the first of October of 1999 Science Daily announced: "Machine demonstrates superhuman speech recognition abilities. University of Southern California biomedical engineers have created the world's first machine system that can recognize spoken words better than humans can." It was referring to a neural network trained by Theodore Berger's team. As far as i can tell, that project has been abandoned and it was never used in any practical application.

In October 2011 a Washington Post headline asked "Apple Siri: the next big revolution in how we interact with gadgets?" Meanwhile, this exchange was going viral on social media.
User: Siri, call me an ambulance
Siri: Okay, from now on I'll call you "an ambulance"

In 2014 the media announced that Vladimir Veselov's and Eugene Demchenko's program Eugene Goostman, which simulated a 13-year-old Ukrainian boy, passed the Turing Test at the Royal Society in London (Washington Post: "A computer just passed the Turing Test in landmark trial"). It makes you wonder what was the I.Q. of the members of the Royal Society, or, at least, of the event organizer, the self-appointed "world's first cyborg" Kevin Warwick, and what was the I.Q. of the journalists who reported his claims. It takes very little ingenuity to fool a "chatbot" impersonating a human being: "How many letters are in the word of the number that follows 4?" Any human being can calculate that 5 follows 4 and contains four letters, but a bot won't know what you are talking about. I can see the bot programmer, who has just read this sentence, frantically coding this question and its answer into the bot, but there are thousands, if not millions, of questions like this one that bots will fail for as long as they don't understand the context. How many words are in this sentence? You just counted them, right? But a bot won't understand the question. Of course, if your Turing Test consists in asking the machine questions whose answers can easily be found on Wikipedia by any idiot, then the machine will easily pass the test.

In 2015 both Microsoft and Baidu announced that their image-recognition software was outperforming humans, i.e. that the error rate of the machine was lower than the error rate of the average human being in recognizing objects. The average human error rate is considered to be 5.1%. However, Microsoft's technology that has surfaced (late that year) is CaptionBot, which has become famous not for its usefulness in recognizing scenes but for its silly mistakes that no human being would make. As for Baidu, its Deep Image system, that ran on the custom-built supercomputer Minwa (432 core processors and 144 GPUs), has not been made available to the public as an app. However, Baidu was disqualified from the most prestigious image-recognition competition in the world (the ImageNet Competition) for cheating. Recognizing images was supposed to be Google's specialty but Google Goggles, introduced in 2010, has flopped. I just tried Goggles again (May 2016). It didn' recognize: towel, toilet paper, faucet, blue jeans... It recognized only one object: the clock. Officially, Google's image recognition software has an error rate of 5%. My test shows more like 90% error rate. In 2015 the Google Photos app tagged two African-Americans as gorillas, causing accusations of racism when in fact it was just poor technology. The media widely reported that Facebook's DeepFace (launched in 2015) correctly identified photos in 97.25% of cases (so claimed Facebook), even causing the European Union to warn Facebook that people's privacy must be protected, but in 2016 it identifies none of my 5,000 friends: it works only if you have a small number of friends.

Sometimes the claims border on ridicule. Let's say that i build an app that asks you to submit the photo of an object, then the picture gets emailed to me, and i email back to you the name of the object: are you impressed by such an app? And, still, countless reviewers marveled at CamFind, the app introduced in 2013 by Los Angeles-based Image Searcher, an app that "recognizes" objects. In most cases it is actually not the app that recognizes objects, but their huge team in the Philippines that is frantically busy tagging the images submitted by users. Remember the automata of centuries ago, that in reality were people camouflaged like machines? In 1769, a chess-playing machine called the Turk, created by Wolfgang von Kempelen, toured the world, winning games wherever it: it concealed a man inside so well that it wasn't exposed as an hoax for many years.

(To be fair, Microsoft's CaptionBot is not bad at all: it was criticized by people who expected human-level abilities in the machine, but, realistically, it exceeds my expectations).

Very few people bother to doublecheck the claims of the A.I. community. The media have a vested interest that the story be told (it sells) and the community as a whole has a vested interest that government and donors believe in the discipline's progress so that more funds will be poured into it.

Paul Nunez in "Brain, Mind, and the Structure of Reality" (2010) distinguishes between Type 1 scientific experiments and Type 2 experiments. Type 1 is an experiment that has been repeated at different locations by different teams and still holds. Type 2 is an experiment that has yielded conflicting results at different laboratories. UFO sightings, levitation tales and exorcisms are not scientific, but many people believe in their claims, and i will call them "Type 3" experiments, experiments that cannot be repeated by other scientists. Too much of Artificial Intelligence falls between Type 2 and Type 3.

News of the feats achieved by machines rapidly propagate worldwide thanks to enthusiastic bloggers and tweeters the same way that news about telepathy and levitation used to spread rapidly worldwide thanks to word of mouth without the slightest requirement of proof. (There are still millions of people who believe that cases of levitation have been documented even though there is no footage and no witness to be found anywhere). The belief in miracles worked the same way: people wanted to believe that a saint had performed a miracle and they transmitted the news to all their acquaintances in a state of delirious fervor without bothering to doublecheck the facts and without providing any means to doublecheck the facts (address? date? who was there? what exactly happened?). The Internet is a much more powerful tool than the old "word of mouth" system. In fact, i believe that part of this discussion about machine intelligence is a discussion not about technology but about the World-wide Web as the most powerful tool ever invented to spread myths. And part of this discussion about machine intelligence is a discussion about the fact that 21st century humans want to believe that super-intelligent machines are coming the same way that people of previous centuries wanted to believe that magicians existed. The number of people whose infirmity has been healed after a visit to the sanctuary of Lourdes is very small (and in all cases one can find a simple medical explanation) but thousands of highly educated people still visit it when they get sick, poor or depressed. On 13 October 1917 in Fatima (Portugal) tens of thousands of people assembled because three shepherd children had been told by the Virgin Mary (the mother of Jesus) that she would appear at high noon. Nobody saw anything special (other than the Sun coming out after a rain) but the word that a miracle had taken place in Fatima spread worldwide. Believe it or not, that is pretty much what happened in 2013 when a fanatical blogger reported a feat performed by an A.I. software or a robot as a new step towards the Singularity. People like me who remain skeptical of the news are frowned upon in the same way that skeptics were looked down upon after Fatima: "What? You still don't believe that the Virgin Mary appeared to those children? What is wrong with you?". Which, of course, shifts the burden of proof on the skeptic who is asked to explain why one would NOT believe in the miracle (sorry, i meant "in the machine's intelligence") instead of pressing the inventor/scientist/lab/firm into proving that the miracle/feat has truly been accomplished and can be repeated at will and that it really did what bloggers said it did.

"Whenever a new science achieves its first big successes, its enthusiastic acolytes always fancy that all questions are now soluble" (Gilbert Ryle, "The Concept of Mind", 1949, six years before Artificial Intelligence was born).

Back to the Table of Contents


Purchase "Intelligence is not Artificial"
Back to Cognitive Science | My book on consciousness | My reviews of books | Contact